Audio, Gifs, and NativeScript Oh my!

It’s been quite a while since my last post. Last year was a busy year with more updates to the The NativeScript Book, but now it’s time to get back to blogging!

For Christmas this year my wife got me a ukulele. Although I’m still terrible at playing I am really excited to learn. The ukulele got me thinking about all kinds of apps I could create to help me learn to play. However, there’s already tons of those so why not have a little bit of fun instead to get back into blogging!

Memory Lane

I grew up in the 90s and the 90s gave us so many great things from Happy Gilmore…

Ok maybe we didn’t really need action figures for everything.

One thing I distinctly remember from my childhood was the dancing sunflower.

Therefore, the only logical connection to my childhood and my new ukulele is to create a dancing flower app!

Audio, Gifs, and NativeScript Oh my!

To create our 1990s dancing flower we only need to do a few things:

• Listen for audio on our device
• A dancing flower gif
• Animate the gif when the audio reaches a certain threshold

To get started I just scaffolded a new NativeScript with Angular project using the tns create command. For the rest of this post we will focus on updating the items component since this will be the quickest way for us to get started.

Audio

The easiest way for us to listen for audio will be to use the NativeScript audio plugin by Brad Martin. All plugins for NativeScript can be easily found in the NativeScript Marketplace. The Marketplace also has NativeScript version compatibility information, installation instructions, and tutorials for each plugin. To install the plugin to your project just use the following CLI command: tns plugin add nativescript-audio

To use the audio plugin, we will create a simple layout in the items component (just replace whatever is already there) with two buttons – one for starting to record/listen and one to stop.

<FlexboxLayout flexDirection="column" width="300">
    <StackLayout height="100%">
            <Button text="Start Listening" class="btn btn-primary" (tap)=onStart($event)></Button>
            <Button text="Stop Listening" class="btn btn-outline" (tap)=onStop($event)></Button>
    </StackLayout>
</FlexboxLayout>

The audio plugin from Brad is easy to get setup, we just need to create an instance of the TNSRecorder class, setup some options, and start recording to a file! This example is setup to run on iOS; in Brad’s sample code he shows how to record to both iOS and Android.

public async startRecording() {
        try {
          if (!TNSRecorder.CAN_RECORD()) {
                console.log('This device cannot record.')
            return;
          }
          const audioFolder = knownFolders.currentApp().getFolder('temp');
          console.log(JSON.stringify(audioFolder));
    
          this.audioFile = `${audioFolder.path}/temp.caf`;
          
    
          const recorderOptions: AudioRecorderOptions = {
            filename: this.audioFile,
            metering: true,
    
            infoCallback: infoObject => {
              console.log(JSON.stringify(infoObject));
            },
    
            errorCallback: errorObject => {
              console.log(JSON.stringify(errorObject));
            }
          };
    
            await this.recorder.start(recorderOptions);
            
          if (recorderOptions.metering) {
            this.setupDancing();
          }
        } catch (err) {
          this.stopDancing();
            console.log(err);
        }
    }

You’ll see in this code that when we start recording we also setup our flower to dance, let’s take a look and see how that works.

Note: To enable recording permissions on iOS you need to update the Info.plist file and on Android you will need to update the AndroidManifest.xml file. Both of these are well document in the installation procedures in the NativeScript Marketplace for the audio plugin.

Dance Party

Since I am not in the least bit artistic, I found the most artistic friend I knew and asked him to make me a dancing flower gif and he delivered (thanks David). Here’s is what he gave me; it is just how I remember it.

The next question is how do we animate a gif in NativeScript? Lucky for use Brad has another plugin for this as well! Another trip to the NativeScript Marketplace and and install from the command line (tns plugin add nativescript-gif).

Adding the flower to our UI is not very difficult. First we update the app.component.ts to import the Gif element so we can use it in our component.

import { Gif } from "nativescript-gif";
registerElement("Gif", () => Gif);

Next we just add the element to our UI in items.component.html.

<StackLayout>
            <Gif src="~/flower.gif" height="100%" id="flower"></Gif>
</StackLayout>

The last piece of the puzzle is to implement use the gif plugin to play the animations when we detect sound just like the real dancing flower and then stop when we don’t hear anything. The audio plugin provides to get a numeric representation of the audio level by calling the getMeters method. This part took a bit of trial and error and may be different for you depending on your device but I settled on anything above -15.0 was what I considered not ambient noise. Here’s what the functions for starting and stoping the gif animations look like:

private setupDancing() {
        this.stopDancing();
        this.removeFile();

        this.dancingInterval = setInterval(() => {            
            let meters = parseFloat(this.recorder.getMeters(0));
            
            if (meters >= -15.0) {
                if (!this.dancing) {
                    console.log('starting dancing');
                    this.flower.start();
                    this.dancing = true;
                }
            } else {
                this.flower.stop()
                this.dancing = false;
            }

            console.log(meters);
            console.log(this.dancing);
        }, 1000)
    }

    private stopDancing() {
        this.flower.stop();
        this.dancing = false;
        
        if (this.dancingInterval) {
            clearInterval(this.dancingInterval);
            this.dancingInterval = undefined;
        }
    }

That’s pretty much all there is too it! Now any time I play my ukulele I can at least play it for someone or something that will enjoy listening to me no matter how terrible I am.