Yolo LPR (License Plate Recognition) Device Based on UNIHIKER

auroraAA Apr 11.2024

0 96 Easy

Introduction

In the current wave of technological advancement, the application of LPR (license plate recognition) technology is becoming increasingly widespread, bringing convenience and efficiency to various industries. The Yolo license plate recognition device based on UNIHIKER is born against this background. This project aims to combine a USB camera with UNIHIKER and utilize the YOLO model to achieve rapid and accurate recognition of license plate numbers. With this device, users can easily realize intelligent monitoring and management of vehicles, improve work efficiency, and enhance security management.

Project Objectives

- Learn how to use the OpenCV library to read video frames and save images.

- Understand the basic principles of LPR (license plate recognition).

- Basic understanding of the YOLO model.

HARDWARE LIST

1 UNIHIKER - IoT Python Single Board Computer with Touchscreen

Link

1 Type-C&Micro 2-in-1 USB Cable

Link

1 USB camera

Link

Software

- Mind+ Programming Software

Practical Process

1/Hardware Setup

Connect the camera to the USB port of Unihiker.

2/Software Development

Step 1: Create and save project files, open Mind+, and remotely connect to Unihiker.

Step 2: Import the license plate recognition folder

Drag the "Chinese_license_plate_detection_recognition" folder into "Files in the Unihiker." See Appendix 1 for the link to the resource folder.

Tip1: Do not modify the folder path; leave it under root.

Tip2: The folder contains files such as the LPR model. The "0-Install_dependency.py" program file is used to install the required dependencies for LPR. "Recognition_plate.py" is a pre-written program module for LPR.

Step 3: Install dependency

Double-click to open the "0-Install_dependency.py" program file in "Chinese_license_plate_detection_recognition," click the Run button in the upper right corner, and wait for automatic installation to complete.

Step 4: Write the program

Create a new project file in the same directory as "0-Install_dependency.py" and name it "1-read_number_test.py".

Sample program:

CODE

import sys
sys.path.append("/root/Chinese_license_plate_detection_recognition")
import Recognition_plate  
import cv2
import numpy as np
cap = cv2.VideoCapture(0)  
cap.set(cv2.CAP_PROP_FRAME_WIDTH, 240)  
cap.set(cv2.CAP_PROP_FRAME_HEIGHT, 320)  
cap.set(cv2.CAP_PROP_BUFFERSIZE, 1)  
cv2.namedWindow('cvwindow', cv2.WND_PROP_FULLSCREEN)
cv2.setWindowProperty('cvwindow', cv2.WND_PROP_FULLSCREEN, cv2.WINDOW_FULLSCREEN)  # 设置窗口为全屏
 
while not cap.isOpened():  
    continue
 
while True:  
    cv2.waitKey(5)  
    cvimg_success, img_src = cap.read()  
    cvimg_h, cvimg_w, cvimg_c = img_src.shape  
    cvimg_w1 = cvimg_h * 240 // 320  
    cvimg_x1 = (cvimg_w - cvimg_w1) // 2
    img_src = img_src[:, cvimg_x1:cvimg_x1 + cvimg_w1]  
    img_src = cv2.resize(img_src, (240, 320))  
    cvimg_src = img_src.copy()  
    cv2.imshow('cvwindow', cvimg_src)  
    if cv2.waitKey(1) & 0xFF == ord('a'):
        cv2.imwrite("/root/Chinese_license_plate_detection_recognition/imgs/3.png", cvimg_src)  # 保存当前的画面
        number = Recognition_plate.recognize()  ###recognize
print(number)

3/Run and Debug

Step 1: Run the main program

Open and run "1-read_number_test.py." You can see the screen showing the live feed from the camera. And now, place the license plate in the camera's frame, press the onboard button 'a' to capture the image for recognition. After a few seconds, you can see the recognized license plate number displayed on the terminal.

You can also find the captured license plate image "3.png" in the "imgs" folder at the same level.

Program Analysis

In the aforementioned "1-read_number_test.py" file, we primarily utilize the OpenCV library to capture images from the camera. And we resize the images to a resolution of 240x320 and display them on the Unihiker screen. Then, we set recognition conditions and, upon pressing the 'a' key, invoke the recognize() function from the Recognition_plate module to identify the license plate.

Open the "Recognition_plate.py" file and we can see that this code segment mainly implements license plate detection and recognition using the Yolo model. The key steps are as follows:

1.Model Loading: We employ the load_model function to load the pre-trained license plate detection and recognition model. The Yolo model is stored in the "models" folder.

2.Image Preprocessing: In the detect_Recognition_plate function, we first preprocess the input image, including operations such as scaling and cropping. Then we convert the preprocessed image to a format acceptable to the model.

3.License Plate Detection: The preprocessed image is inputted into the license plate detection model, yielding the position and keypoint coordinates of the license plate.

4.Obtain License Plate Image: We use the key point coordinates of the license plate and use the four_point_transform function to perform a perspective transformation to get an image of the license plate.

5.License Plate Recognition: We input the license plate image into the license plate recognition model to get the license plate number. If the color recognition option is set, the color of the license plate can also be obtained.

6.Result Processing: We organize the detection and recognition results, including the position of the license plate, the license plate number, and the color of the license plate.

7.Result Display: We use the draw_result function to draw the results of detection and recognition onto the original image, including drawing the location of the license plate and the license plate number.

Knowledge Domain

1/Principles of License Plate Recognition

The principle of license plate recognition involves capturing images of vehicles through cameras and subsequently identifying the license plate numbers from these vehicle images.

The recognition process involves the following steps: vehicle detection——image acquisition——image preprocessing——license plate localization——character segmentation——character recognition——result output.

Vehicle Detection: Before taking a picture we need to detect whether the vehicle is in the best focus position of the camera. So vehicle detection is the first step in license plate recognition. The camera is passive to collect data, so it doesn't know when to take pictures. In fact, there are many ways to trigger the camera to take pictures, for example, the most commonly used in life is the infrared method. When the vehicle enters the shooting area, the infrared rays will be blocked by the vehicle, this time the camera and the system linkage, it will take pictures of the vehicle.

- Vehicle Detection: Before taking a picture we need to detect whether the vehicle is in the best focus position of the camera. So vehicle detection is the first step in license plate recognition. The camera is passive to collect data, so it doesn't know when to take pictures. In fact, there are many ways to trigger the camera to take pictures, for example, the most commonly used in life is the infrared method. When the vehicle enters the shooting area, the infrared rays will be blocked by the vehicle, this time the camera and the system linkage, it will take pictures of the vehicle.

- Image Acquisition: The step of taking a photo of a vehicle that enters the effective focal range is easy to understand. The software side of the license plate recognition system will intercept the picture based on the video of the vehicle's entry, or take the picture directly. After obtaining the photo, it is provided to the recognition system for backup.

- Image Preprocessing: We all know that cars have many colors, and when there are many colors, computers are easily dazzled. Therefore, we need to process the picture only in black and white. This picture processing is the picture binarization. During the process, each pixel of the image will be judged by its RGB value, and according to the set value (e.g. 160), pixels larger than 160 will be processed as white, and pixels smaller than 160 will be processed as black.

- License Plate Localization: Once the image has been processed, how can we locate the license plate? The license plate is a regular rectangle. Then we need to find the rectangle in the picture. Then the computer scans the processed picture and records all the pixels with the color from black to white or white to black from left to right and from top to bottom. These pixels are then used to calculate which area is a rectangle and to locate the position that matches the proportions of the license plate.

How to tell if it's a license plate number? It's simple, another scan of the area. Since it's a processed image, if there's a license plate number. There will definitely be black and white changes, especially in the vertical direction. This narrows down the scope and enables you to find the license plate quickly.

- Character Segmentation: After we have found the license plate, we need to do a vertical segmentation of each character width based on the scanned license plate. Since the license plate image has been processed into black and white colors, it is easy to get the height and width of the characters because the characters are either white characters on a black background or black characters on a white background. The segmentation is based on this, all the characters of the license plate are divided into individual characters.

- Character Recognition: We need to compare the segmented characters sequentially from the license plate library templates and then find the image with the highest similarity from the license plate template library. Purpose is to obtain the Chinese characters of the license plate province, 26 English capital letters, and 10 numbers from 0-9.

- Result Output: During the comparison process, the image templates with the highest similarity are stored into an array in order, and finally the successfully recognized license plate numbers are obtained by traversing the method.

2/Introduction to YOLO

YOLO (You Only Look Once) is a popular object detection algorithm that differs from traditional methods by unifying bounding box prediction and class probability prediction into a single network. This enables end-to-end object detection, maintaining high accuracy while achieving high-speed operation.

YOLO divides the input image into an SxS grid, with each grid responsible for predicting an object. Each grid predicts B bounding boxes and their confidences, along with C class probabilities. The confidence of a bounding box refers to the product of the confidence that there is an object within the predicted bounding box and the confidence that the bounding box accurately predicts the object's position. If there is no object in the grid, the confidence of the predicted bounding box in that grid should be zero. Otherwise, the confidence should be equal to the Intersection over Union (IoU) of the predicted bounding box and the ground truth bounding box.

The main advantages of YOLO are its speed and high accuracy. Since it does all the predictions in a single network, it can perform object detection in real time. Additionally, by directly predicting bounding boxes in a global context, it can handle objects of various scales and shapes well. However, YOLO has some limitations, such as it may not perform well with small objects and predicting large numbers of overlapping objects.

Currently, YOLO has multiple versions, including YOLOv1, YOLOv2 (YOLO9000), YOLOv3, YOLOv4, and the latest YOLOv5. Each new version makes improvements in speed and accuracy.

Appendix 1: Material and Additional Program