Barcode Recognition-Based Self Checkout System

1 Project Introduction

In large supermarkets or unmanned convenience stores, we often encounter such scenarios: customers simply place products in the recognition area, and the system automatically identifies product information, calculates the total price, and completes payment. This process requires almost no manual operation, making it fast and convenient. This intelligent checkout method is a typical application of the "Product Adding System" in reality. It transforms the shopping experience from traditional manual scanning into a new paradigm of intelligent recognition and automatic accounting. Behind it lies the combination of image recognition and information processing technologies, enabling machines to "understand" products and automatically complete the adding and checkout operations.

In this project, we will simulate the working process of this real-world system. We will use HuskyLens 2 to recognize barcodes of different products and UNIHIKER K10 to record, calculate, and display product names and quantities. When a product is recognized, the screen will update the information in real-time and voice broadcast the result, making the adding process more intuitive and intelligent. By building the "Product Adding System" hands-on, we can gain a deeper understanding of the working principles behind intelligent retail and experience the huge potential of artificial intelligence in changing shopping methods.

Demo Video

2. Project Implementation Principle

In Barcode Recognition mode, HuskyLens 2 parses the image captured by the camera using its built-in AI algorithm, extracts barcode information from it, and converts it into unique ID data. The UNIHIKER K10 establishes a connection with HuskyLens 2 via the I2C communication protocol, periodically sends recognition requests to HuskyLens 2, and after receiving the ID data, UNIHIKER K10 accumulates the quantity of corresponding products based on a predefined ID-product type mapping relationship (e.g., ID1 corresponds to kiwifruit, ID2 corresponds to oranges). Meanwhile, the UNIHIKER K10 displays the current product quantity in the designated screen area in real-time, and when Button B is pressed, it calculates the total price using a formula, uses the text-to-speech module to broadcast "Total price is XX yuan", and provides users with immediate settlement feedback.

3. Hardware and Software Preparation

3.1 Equipment List

Software: Mind+ Graphical Programming Software

3.2 Hardware Connection

Make connections by referring to the diagram below.

3.3 Software Preparation

Download and install the Mind+ installation package (Version 2 and above)  from the official website. Double-click to open it after installation.

4. Project Making

4.1 HuskyLens 2

First, select the protocol type for HuskyLens 2.  
Tap System Settings -> Protocol Type -> Select I2C communication mode, then return to the main menu interface.

Second, swipe the screen to find the "Barcode Recognition" function.

Aim HuskyLens 2 at the scene containing the barcode. When a barcode is detected, the screen will frame all barcodes in the image with white boxes and display the information contained within the barcodes.

After configuring the parameters and selecting the mode in HuskyLens 2, next, train the model with multiple product barcodes to trigger the product adding operation. Follow these steps:

Align the barcode to be learned, adjust the angle of HuskyLens 2 to capture the barcode, center the "cross" in the white box, then press Button A in the top-right corner of HuskyLens 2 to learn this barcode.

Once learned, if a previously learned barcode is recognized, the screen will frame the barcode with a colored box and display "name: IDx" above the barcode, where "name" defaults to the information in the barcode, e.g., "www.enjson.com:ID1". Repeat this process to learn more barcodes.

For more detailed usage of HuskyLens 2, please refer to the following URL:
https://wiki.dfrobot.com/_SKU_SEN0638_Gravity_HUSKYLENS_2_AI_Camera_Vision_Sensor

After learning products’ barcodes in HuskyLens 2, you can start writing code to implement the barcode recognition-based self-checkout system.

4.2 Programming

Open the programming software Mind+, choose "Coding" mode, then click "Upload" to create a new project.

Next, add the required extensions in Mind+, including UNIHIKER K10 and HuskyLens 2.
Enter the "Extensions" page, switch to "Board" tab, search for"K10", and click "UNIHIKER K10".

Load the "HuskyLens 2 AI Camera" library by the same way from "Module" page.

Click the "Back" button to return to the programming interface.

Click the "Connect Devices", choose your device and "Connect".

After the device is successfully connected, write the program as follows:

The analysis of the core code is as follows:

There is a complete program file for this project in the attachment. (Note: The .mp file is compatible with both Mind+ v1.x (e.g., v1.8.1) and v2.x (e.g., v2.0), while the .mpcode file only works with Mind+ v2.0)
Open Project->Open Local File to load project.

Select the project in the attachment and click "Open".

Click "Upload" to run the program.

The effect is as follows: