Edge Impulse Empowered: Exploring TinyML Projects with Arduino, ESP32, and Raspberry Pi

When discussing machine learning, what immediately springs to mind are concepts like machine vision, large language models, RNNs, CNNs, and algorithms such as SVMs and decision trees. Typically, these processes are computer-based. However, are you aware of the multitude of machine learning applications that surround us daily?

Take the Apple Watch, for instance. It boasts a fascinating feature - handwashing detection. This function automatically recognizes when you're washing your hands and diligently records the duration. The latest watchOS update has introduced a gesture control that involves pinching and clenching your fist, which is truly remarkable. These are prime examples of machine learning applications in our immediate environment. The implementation of machine learning in wearable devices is a testament to the marvels of technology.

Today, let's delve into the world of TinyML.

TinyML, or Micro Machine Learning, is a specialized niche within the broader realm of machine learning. Its focus lies in deploying machine learning applications on devices constrained by limited resources and capabilities. The essence of TinyML is to bring machine learning to the edge in a profoundly efficient manner, enabling battery-powered, microcontroller-based embedded devices to perform machine learning tasks with real-time responsiveness.

The application scope of TinyML is remarkably extensive, encompassing areas such as:

· Wearable Devices: The handwashing detection and gesture recognition on the Apple Watch exemplify TinyML in action.

· Industrial IoT: TinyML is instrumental in fault detection and predictive maintenance in industrial equipment.

· Medical and Health: TinyML finds applications in diagnostic and monitoring tools in medical devices.

· Smart Cities: It contributes to traffic monitoring and environmental surveillance in urban settings.

The advantages of TinyML are primarily evident in the following aspects:

· Lower Latency: Data processing occurs on the edge devices, eliminating the need to transmit data to servers for inference.

· Energy Efficiency: The microcontrollers, like the ESP32 series, consume minimal power, allowing extended operation without frequent charging.

· Reduced Bandwidth: Inferences require little to no internet connectivity.

· Data Privacy: Data remains on the edge device, ensuring privacy since it is not stored on servers.

The successful implementation of TinyML hinges on a holistic design encompassing hardware, software, and algorithms.

· Hardware-wise, it necessitates low-power microcontrollers such as the ESP32 series.

· In software, it requires frameworks specifically designed for TinyML, like TensorFlow Lite and MicroPython.

· Algorithmically, it utilizes machine learning algorithms optimized for TinyML, including Convolutional Neural Networks (CNNs) and tree-based methods.

As a rapidly evolving field, the future of TinyML looks promising, with expanding applications thanks to ongoing advancements in hardware, software, and algorithms. Next, I will share with you a few quintessential applications of TinyML.

Electronic Nose – The Application of Gas MEMS Sensors

Odor, a mysterious element, is something everyone experiences, whether it's the fragrance of flowers or an overpowering stench. It influences our emotions, memories, and appetite, eliciting feelings of happiness, sadness, anger, or fear. Odors can evoke memories of people or places and affect our appetite significantly.

Odors are caused by volatile compounds released into the air from objects. These compounds enter our nostrils and are detected by olfactory organs. These organs relay the compounds' signals to the brain, which interprets them as distinct smells.

The perception of odor is subjective; everyone experiences scents differently. While some may find certain odors pleasant, others may not. People's sensitivity to smells also varies significantly.

Odors play a crucial role in our lives, helping us understand our surroundings and connect with others. However, enabling machines to detect and interpret odors has always been challenging. Now, with advancements in electrochemistry, we are beginning to see, through the lens of TinyML, how machines can detect odors and help us improve our lives.

In his project "Building an Electronic Nose (Olfactory) with MEMS Gas Detection Sensor and Edge Impulse Platform," RiverTrue leveraged an ESP32 and MEMS electrochemical gas sensors. With the assistance of the Edge Impulse platform, he trained a model capable of distinguishing between juice and alcoholic beverages. This exploration revealed that a combination of gas sensors could effectively differentiate between alcoholic drinks and juices, showcasing the potential of MEMS sensors in nuanced odor detection.

Additionally, they trained another model to predict the storage time of fruits, particularly focusing on the degree of spoilage in mangoes during warehousing. This training significantly enhanced the understanding of potential spoilage levels.

For those interested in this project, more detailed information can be found at the following link:

"Building an Electronic Nose (Olfactory) with MEMS Gas Detection Sensor and Edge Impulse Platform - D."

There's also another classic electronic nose project worth exploring, available at the link:

"How I Built a Connected Artificial Nose (and How You Can Too!) - Benjamin Cabé."

This project, completed by Benjamin Cabe, underscores the remarkable capabilities of TinyML in scenarios involving a combination of various sensors, showcasing its incredible potential.

Electronic Eye – The Wonder of Machine Vision

Despite having access to powerful devices like personal computers with graphics cards, which effortlessly handle vision-based inference like YOLO, such equipment isn't always the most practical choice for IoT transformations in devices like water meters or pressure gauges. I once aspired to install a camera on my water meter to record daily water usage, but the programming complexity made me abandon the idea. However, with the advent of TinyML, I was pleasantly surprised to find others who shared my vision of retrofitting their analog meters with cameras.

Brandon Satrom from Texas, aiming to efficiently control the filtration and pressure pump of his home swimming pool, crafted a monitor using a Raspberry Pi 4 and a Raspberry Pi camera. He employed TinyML to learn and train a model capable of recognizing the needle and dial readings on an analog pressure gauge. This innovation spared him the hassle of monitoring the gauge under the blazing sun to control the water pump.

He continued to utilize the Edge Impulse platform for training his model. Remarkably, he completed the model training in just a few hours and successfully deployed it on his Raspberry Pi 4.

This was previously unimaginable and would have required extensive engineering work. You can delve into the specifics in the article titled "Monitor the Analog World With TinyML."

Additionally, for insights on how to achieve similar analog gauge detection using a more cost-effective ESP32 with a camera, you can refer to the article at petewarden.com.

Electronic Ear – The Extended Range of Hearing

The concept of the electronic ear, rooted in traditional sound capture technology, has seen its applications and capabilities greatly expanded with recent technological advancements. Equipped with highly sensitive sensors, such as MEMS microphones, these devices can capture faint sound waves and distant sound sources. With the integration of machine learning and artificial intelligence, electronic ears now not only collect sounds but also analyze and identify them. For instance, in environmental monitoring, electronic ears can detect various natural sounds to identify potential environmental issues, like changes in wildlife migration patterns. In urban management, they monitor traffic noise, aiding in the optimization of city layouts and traffic flow. These applications demonstrate how electronic ears have become powerful tools in understanding and interacting with our world.

A project I discovered on GitHub, featuring a hydrophone, showcases the capabilities of an electronic ear. The creator used TinyML to record and learn the sounds of boats in water, resulting in a waterborne sound classification model for boats, effectively distinguishing between different types of vessels.

The author ingenatively designed a custom hydrophone expansion board, utilizing an Arduino as the main controller to create a hydrophone device. By employing the Edge Impulse platform, a series of underwater sounds were recorded, tagged, and trained, leading to the successful development of a machine learning model capable of distinguishing boats. This model operates directly on the Arduino.

For more comprehensive details, you can visit the following link: [Hydrophone Project on GitHub](https://github.com/IQTLabs/AISonobuoy/tree/main/Hydrophone).

Electronic Sensation – Gesture Recognition

Human sensory experiences are remarkably rich. We can feel temperature changes, the sensation of dizziness, and the touch of a surface – these are all astonishing senses. The amount of work done to replicate these sensations in technology is immense and too vast to cover in a single article. Therefore, for simplicity, I will focus on motion perception. As mentioned earlier, Apple introduced an interaction method for its watches using finger pinching gestures. This motion sensing not only utilizes gyroscope data but also integrates signals from heart rate sensors, inferring results with the help of a vast amount of training data and artificial intelligence chips.

Many wearable devices can adopt this method for gesture recognition. This innovative human-computer interaction relies on high-precision sensors and advanced data processing algorithms to accurately identify and interpret human gestures and movements. The essence of this technology is to transform human actions into electronic signals, then analyze and understand them through machine learning and pattern recognition techniques.

MEMS sensors are particularly crucial in this field, widely used in wearable and mobile devices due to their small size and low energy consumption. These sensors can detect subtle gesture changes, like slight finger movements or palm rotations, converting these physical actions into digital signals. With tinyML technology, these devices can process this data in real-time without relying on high-performance computing resources, providing accurate gesture recognition.

Gesture recognition technology has significant applications in various fields. In smart homes, users can control various intelligent devices, such as lights, speakers, and even curtains, with simple gestures. In virtual reality (VR) and augmented reality (AR), this technology offers a more natural and intuitive way of interaction, enhancing the immersive experience. Additionally, gesture recognition plays a vital role in assistive technology, aiding individuals with communication or physical limitations in effectively interacting and controlling electronic devices.

Bangle.js is an open-source smartwatch offering a hackable approach for users to play with and personalize their watch. In the link provided, they showcase a method using TinyML for gesture learning. This allows you to control watch functions with gestures trained by yourself, enabling a highly customizable and interactive experience.

Following this tutorial, you might even attempt to replicate the Apple Watch's finger pinching or fist clenching control methods.

Here is the link to their specific implementation method:

"Bangle.js and Edge Impulse for Machine Learning - Espruino."

If you find the above method challenging, I also recommend checking out this more basic tutorial based on ESP32 and TinyML:

"Gesture Classification with Esp32 and TinyML."

Conclusion

With the rapid advancement of TinyML technology, we can now implement complex machine learning applications on a variety of miniature devices. As demonstrated by the five TinyML projects discussed above, this technology has penetrated every aspect of our daily lives: from electronic tongues monitoring water quality to smartwatches recognizing gestures, from electronic noses analyzing odors to electronic ears identifying environmental sounds, and the application of machine vision in small devices. These innovations showcase TinyML's immense potential in edge computing and real-time data analysis.

Furthermore, TinyML's applications extend beyond portable and low-power devices, playing a crucial role in data privacy and reducing energy consumption. By processing data locally, TinyML minimizes the need for data transmission, thus lowering energy consumption and protecting user privacy. This is particularly important for applications with strict requirements for data security and energy efficiency.

Additionally, TinyML has invaluable applications in the industrial sector, with equipment fault detection and predictive maintenance being crucial.

In summary, TinyML is opening up a new world, enabling us to interact with our surroundings in unprecedented ways. As technology continues to advance, we can expect TinyML to bring more innovations and changes, making our lives smarter and more convenient.

If you find such articles interesting, feel free to follow, comment, and share. Suggestions for content you'd like to see are also welcome in the comments.