Punchcards recognition with Machine Learning and LattePanda IOTA

As a tribute to punched cards and to test the LattePanda IOTA mini PC, this project recognizes visually “punched” cards — printed and tagged here instead of punched — using a  USB camera and a local machine learning model.

The cards

I have created a one row card layout with encoded ASCII letters. 

Examples:

• A 01000001
• B 01000010
•  

For each letter, 10 variations with the “holes” in different positions.

LattePanda IOTA

The LattePanda IOTA is an x86 mini PC SBC with enough performance to run traditional robotics workloads and on-device AI inference. It features an Intel Processor N150 (4C/4T), 8GB or 16GB of LPDDR5 memory, 64GB or 128GB of eMMC storage, and an onboard RP2040 microcontroller for real time sensors and actuators.

Besides the board, for this project you need a standard USB web cam, the active cooler and a Power Supply.

Machine Learning model training

All the card images were uploaded to a new Edge Impulse project using Data Acquisition, with one label per image and an 82/18 split between Training and Testing.

An Impulse with a Classification Learning block and Grayscale color depth achieved perfect recognition, with only 50 training cycles and a 0.0005 learning rate:

You can sign up for free at EdgeImpulse and clone this project https://studio.edgeimpulse.com/studio/863714 

LattePanda Setup

The LattePanda IOTA originally comes with Windows pre-installed on the eMMC, so I replaced the OS by Ubuntu LTS. After flashing the Ubuntu image to a USB stick with balenaEtcher, the device was booted from USB by pressing F7 and selecting the USB pen drive.

Commands to execute after installing Ubuntu:

sudo apt update

sudo apt install openssh-server -y
sudo systemctl enable ssh
sudo systemctl start ssh

sudo apt install curl -y
curl -fsSL https://deb.nodesource.com/setup_lts.x | sudo -E bash -
sudo apt install nodejs -y

sudo npm install -g edge-impulse-linux –unsafe-perm
sudo npm install -g edge-impulse-cli –unsafe-perm

Now that everything is installed, you can execute: edge-impulse-linux-runnerLog in, select the Edge Impulse project (if you have more than 1), and inference output begins immediately.For example:

classifyRes 1ms. {  A: 0.9214,  B: 0.4301,  C: 0.2434,  E: 0.1809,  H: 0.1588,  L: 0.0557,  O: 0.0096 }

Values represent probabilities for each class.Live visualization is also available at port 4912 on the IP address of the LattePanda on your network. For example, http://192.168.0.100:4912. This is helpful if you need to adjust the web cam position or punch card position.

Python parser

I have also included a Python script to execute the runner, capture the output, parse the result block and display the most probable character.After cloning the GitHub repo linked above, you can launch the runner with:python3 runner.py

More details at https://docs.edgeimpulse.com/projects/expert-network/cv-punchcards-lattepanda

Link to the source code https://github.com/ronibandini/PunchedCards

Final notes

The DFRobot LattePanda IOTA was easy to setup with Edge Impulse. While this project is a tribute to punched-card computing, the same principles apply to recognizing visual states in control panels, tags, physical tokens, or other symbolic markers captured by a camera.