Pose Classification-Based Face-Tracking Oscillating Fan

1.1 Project Introduction

This project aims to create an intelligent fan that rotates while follows human face movements. Unlike ordinary oscillating fans, our fan can detect facial key points in real-time to automatically track face positions.

Of course, you can use it to continuously send cool air to your family. Placing it on a workbench allows it to provide cool air by following your face position. If you replace the fan module with another module (such as a camera), you can also turn the device into a face-tracking camera simulator. This project comprehensively demonstrates the easy practice of AI technology in daily life from data collection, model training, to inference and application, making pose classification technology accessible, interactive, and more interesting!

1.2 Demo Video

2.Project Implementation Principle

This project implements a posture classification-based oscillating fan, covering the entire process from data preparation to model deployment and application. Specifically, it first collects and annotates posture images through the Mind+ model training platform to build a dataset, then trains a posture classification model. After training, it realizes real-time inference and outputs human face positions on the computer. Finally, the UNIHIKER K10 receives the inference results through the IoT platform and controls the servo to rotate, driving the fan to oscillate.

3.Hardware and Software Preparation

3.1 Equipment List

3.2 Hardware Connection

Make connections by referring to the diagram below.

3.3 Software Preparation

Download and install the Mind+ installation package(Version 2 or above) from the official website. Double-click to open it after installation.

Double-click "start SIoT.bat" (which you can find in the attachment) to run the program and get your IP address (e.g., 192.168.9.231).

Note:
When the system pops up the window "Do you want to allow public and private networks to access this app?" for the first time, click "Show more", check both "Public networks" and "Private networks", then click "Allow". Otherwise, devices may fail to access the server.
This program needs to keep running during the data transmission.

4.Project Making

We use Mind+'s model training to train the face position recognition model.

First, open the Mind+ software, click on "Model" and select "Pose Classification" (Note: Only Mind+ 2.0 and above versions have the model training function).

The initial interface will open as follows:

The page is divided into three sections, from left to right: Data Collection, Model Training, and Model Validation & Deployment.

4.1 Data Collection and Labeling

The first step in model training is to prepare the pose image database. In this project, we use left, right, and middle to label the head positions.

Follow the steps below to collect and label data using the camera.

Click the "Camera" button to start data collection. This button directly opens the built-in computer camera for data collection.

Press and hold the "Hold to Record" button to start data collection, changing angles to record multiple images of the target object. By default, twenty photos is collected per second, which can be adjusted in the settings if needed.

Tip: FPS, which means Frames Per Second, refers to the number of image frames collected per second.

After collecting all data of this class, rename the "Class1" to "left".

Follow the same method to complete the image collection and labeling for "right" and "middle".

4.2 Model Training

Click "Train Model" and wait for the model training to complete.

Tip: To adjust training parameters, click "Advanced Settings".

4.3 Validation

After the model training is completed, model validation can be performed to check the model performance. Camera-based validation allows direct observation of the model's application effect. Please follow these steps:

Enable the "Input" switch → Select the camera → Aim the camera at the target → Observe the output

4.4  Inference Result Push

During validation, the inferred pose data can be pushed to the IoT platform (SIoT platform) in real-time, and will be transmitted to UNIHIKER K10 via the IoT. The specific operations are as follows:

Step 1: Click the red dot next to "Real-time Result Push" to open the server settings window.

Step 2: Fill in the server parameters, select "SIoT V2" as the server, and set the MQTT Server Address to the local IP address you recorded earlier.

Step 3: Click "Complete", and the red dot will turn green, indicating that data is pushed in real-time.

4.5 Programming

Now, we can start writing the UNIHIKER K10 program to receive inference results and control the servo's rotation.

Step 1: Click "+", then select "Upload Mode" in the "Coding" tab.

Step 2: Connect device.

Step 3: Download and load relevant extension libraries.

Since we need the UNIHIKER K10 to receive data and control the servo, we need to load the UNIHIKER K10 library, Wi-Fi, MQTT, and Servo libraries. The search method is as follows:
UNIHIKER K10 library: Click "Extensions", in "Board", search for "UNIHIKER K10" and click to download. Wait for the download to complete, then click again to finish loading.

Wi-Fi, MQTT, and Servo libraries: Still on the "Extensions" page, select "Module" and click to download. Wait for the download to complete, then click again to finish loading.

Step 4: Write the program
Write or directly open the attached program. The method to open the project is shown in the following figure:

The opened program is shown in the following diagram:

The analysis of the core code is as follows:

Step 5: Click the "Upload" icon to run the program and wait for the program to finish uploading.

Stay in the Pose Classification page and move left and right in front of your computer’s camera, observe the rotation of the servo horn, and adjust its installation position. Then, using a basic structure, you’ll be able to experience the face-tracking oscillating fan.

The implementation effect is shown in the following image:

5. Attachment

SIoT V2_EN: https://drive.google.com/file/d/1jOcsHETgLeWd7lVjnL0qlSfMyhJ8DSuJ/view?usp=sharing