TensorFlow Lite and Micro_speech

By Anthony Aiyedun

Intro to TensorFlow Lite

My Arduino Nano 33 came early. Let's get started

I started by installing Arduino IDE for Windows and followed that up by installing the TensorFlow Lit Micro Library on GitHub. It is important to note that it is necessary to have the git path installed to get the TensorFlow library, on Windows. The TensorFlow lit library comes with 4 examples codes:

• Hello_world

• Magic_wand - gesture recognition using the onboard IMU

• Micro_speech - speech recognition using the onboard microphone

• Person_detection - person detection using an external ArduCam camera

All of these examples were created based on previously trained data. I will be using the Micro_speech example to get an understanding of the training progress and see it in practice.

Figure 1 . Micro_Speech code on Arduino IDE

As seen above in Figure 1, I installed TensorFlow and opened the Micro_speech example, and ran it. Below is a video showing the code working.

https://video.wixstatic.com/video/964742_e2f1c0c0d5cb420baab197a124b0d8ae/1080p/mp4/file.mp4

Breaking Down Micro_Speech

As seen in Figure 1, there are multiple different pages of code within the main Micro_Speech program, such as Arduino_main.cpp, audio_provider.h, feature_provider.h, and so on. In the context of machine learning, the main project files we wish to focus on are:

• micro_features_model.cpp - this model has been trained on a dataset of the audio recordings of the "yes and no" sounds.

• recognize_commands.cpp - this file contains important code used to interpret the sound coming from the Arduino's microphone.

The micro_features_model.cpp is a TensorFlow FlatBuffer Lite file that has been converted into a C data array. This model can classify images into 1000 different classes.

The recognize_commands.cpp is split into two major parts, the RecognizeCommands class, and the ProcessLatestResults class. To simplify it: the RecognizeCommands class takes audio data and uses a machine-learning model to classify it into different categories. The ProcessLatestResults method is called to process the latest results from the machine learning model and return the category of the audio data.