1 Project Introduction
1.1 Demo Video
1.2 Project Design
Have you ever wondered about your sleep quality at night? Have you wanted to know if you
Chinese herbal medicine is a treasure of Chinese culture, but for many enthusiasts and beginners, dried medicinal materials often have similar appearances and are hard to identify. To overcome this cognitive threshold, this project innovatively integrates the Mind+ 2.0 training tool with the UNIHIKER M10, using dandelion and honeysuckle image classification as an example, to create an intelligent Chinese herbal medicine recognizer.
Through this project, you will learn about:
• Model Training: How to use the Mind+ 2.0 training tool to train an image classification model.
• Model Deployment: How to deploy the trained model on the UNIHIKER M10 to achieve Chinese herbal medicine recognition.
• Ethical Discussion: How to think about and address AI ethical issues such as data bias and security boundaries in technical practice.
1.3 Project Implementation Process
This project trains a Chinese herbal medicine image classification model via the image classification module in the Mind+2.0 model training tool and deploys the model to the UNIHIKER M10 for intelligent recognition. The overall project flow is shown in the following figure:
How to Train a Model
The model training process generally consists of three steps: "Data Collection and Labeling", "Model Training", and "Model Validation".
• Data Collection and Labeling:
Capture and collect images of dandelions and honeysuckles, then perform preprocessing like cleaning, labeling, and classification.
• Model Training:
Input the prepared dataset into a suitable model for training to identify feature patterns among images of different categories.
• Model Validation:
Use untrained test images to evaluate the model and observe its recognition accuracy.
How to Deploy a Model
Model deployment is the process of exporting a trained and validated model from the Mind+ Model Training tool and running it on the target hardware to enable model inference applications.
Real-time result pushing is another model deployment approach. Unlike exporting the model to run on the hardware side, real-time result pushing performs inference on the platform side and instantly transmits the prediction results to the IoT server or external device.
2 AI Knowledge Garden - Image Classification
2.1 Image Classification
Image classification is a type of pattern classification that distinguishes between different categories of images based on the shapes of objects or scene features within the images, such as cat and dog classification.
2.2 Challenges in Image Classification
Despite the remarkable advancements in image classification technology, there remain significant challenges in enabling machines to "see" the world as humans do:
Viewpoint Variation: The same object appears drastically different when viewed from various angles.
Lighting Conditions: Extreme light, dim light, and shadows can all alter an object's appearance.
Scale Variation: Objects within an image can vary drastically in size, from very large to very small.
Occlusion: Objects might be partially blocked, revealing only a small portion of their structure.
Background Interference: Background elements can closely resemble the target object, making it difficult to distinguish.
Intra-class Variability and Inter-class Similarity: Objects within the same category often exhibit significant visual differences (e.g., different breeds of dogs), while distinct categories may appear very similar (e.g., leopards and cheetahs).
3 Chinese Herbal Medicine Model Training
3.1 Download Software and Create Training Project
Download and install the Mind+ 2.0 or higher version installer from the official website, then double-click to open it after installation.
Creating a New Project，Click "Model" in the left navigation bar and select the "Image Classification" task.
3.2 Data Preparation
• Label Setup
• Data Collection
Data can be collected either by shooting on-site via the camera or by uploading existing data.
It is recommended to place the herbal samples on a white background in a well-lit environment, capturing images from different angles and distances. Prepare 80-120 high-quality images for each category.
3.3 Model Training
• Model Training
Click the "Train Model" button to start the training process.
• Training Parameter Settings
Click the "Advanced Settings" button to configure parameters.
• Training Process and Result Observation
During training, click "Learn More" in "Advanced Settings" to view the training logs.
3.4 Model Validation
• Single Image Testing
Select "File," click "Click to Upload File," and choose an untrained image for validation.
• Real-time Testing
Select "Webcam" to perform real-time classification.
• Model Optimization and Retraining
If the validation results are unsatisfactory, retrain the model by improving data quality, adjusting model parameters, or other methods.
4 Chinese Herbal Medicine Model Deployment
4.1 Deployment to UNIHIKER M10
UNIHIKER M10 System Version Upgrade Tutorial:
https://www.unihiker.com/wiki/SystemAndConfiguration/UnihikerOS/unihiker_os_burn/
Model Export
Click the "Export Model" button to export the model package to your local computer.
Programming Environment and Extension Setup
For specific steps, please refer to the Mind+ 2.0 Basic User Guide：
https://mindplus.dfrobot.com/mp2/AITools/Basic_description/model_deployment/model-deployment/#522-programming-environment-and-extension-preparation.
Model Inference and Application
• Upload Model
Upload the exported ONNX file and YAML configuration file to the target environment or hardware platform.
• Program Writing
When a dandelion is recognized, display the prompt text "Dandelion" on the UNIHIKER M10 screen;
When a honeysuckle is recognized, display the prompt text "Honeysuckle" on the UNIHIKER M10 screen.
• Run and Verify
4.2 Deploy to UNIHIKER K10
Model Deployment Approach
With a real-time push deployment approach, the AI model training platform pushes model inference results to the IoT server in real time. The UNIHIKER K10 receives information from the IoT server, retrieves inference results, and displays the corresponding knowledge cards on screen.
Programming Environment and Extension Setup
For specific steps, please refer to the Mind+ 2.0 Basic User Guide：
https://mindplus.dfrobot.com/mp2/AITools/Basic_description/real_time_push/real-time-push/
Program Example
Write the Program
The UNIHIKER K10 receives real-time results pushed to MQTT and analyzes the MQTT messages.
• If the received MQTT message is "Dandelion", the UNIHIKER K10 screen displays "Detection Result: Dandelion" and shows the corresponding knowledge card.
• If the received MQTT message is "Honeysuckle", the UNIHIKER K10 screen displays "Detection Result: Honeysuckle" and shows the corresponding knowledge card.
• Run and Verify
5 AI Ethics Discussion
Technology itself is neutral, but technology designers and users bear responsibility. While enjoying the convenience AI brings, we need to actively think about the ethical issues behind it and explore responsible solutions. Now, let's discuss the core ethical challenges in the Chinese herbal medicine recognition project.
5.1 Data Bias and Model Limitations
• Problem:
What if the model's recognition ability decreases with differences in the morphology of medicinal herbs from different origins, seasons, and growth stages?
• Solution:
Build a Diverse Dataset: The coverage of training data determines the model's cognitive boundaries. Continuously collect and integrate images of medicinal herbs from different origins, growth stages, and processing techniques to establish a more comprehensive and representative dataset, thereby alleviating data bias from the root cause.
5.2 Misjudgment Responsibility and Safety Boundaries
• Problem:
Model misrecognition may lead to medication errors and health risks. How to deal with this?
• Solution:
Introduce Confidence Evaluation Mechanism: Integrate confidence output functionality into the deployed model. When the model's confidence in the recognition result is below a preset threshold (e.g., 90%), the system should actively issue a clear warning of "Recognition Suspected" and recommend manual review by a professional pharmacist.
6 Self-Test
6.1 Extended Exercise
Collect image data on other topics, create a new classification model in Mind+, such as a three-classification model for Ragdoll cats, Orange cats, and Siamese cats, and deploy it to the UNIHIKER M10 or K10 for recognition testing.
6.2 Learning Evaluation Form
Attachment
Google Drive: https://drive.google.com/file/d/1mLJJkU_c_Juy3dg_hE-KVI2YFqiIONpn/view?usp=sharing
