Use Your Hand as the Controller — Experience a New Way to Play Angry Birds with HUSKYLENS 2

The HUSKYLENS 2 AI Vision Sensor is an advanced embedded vision module powered by the Kendryte K230 processor, delivering up to 6 TOPS of dedicated AI inference performance. It features over 20 built-in AI models, including face recognition, object recognition, pose recognition — all ready to use out of the box.

In addition to these built-in models, HUSKYLENS 2 supports a complete custom model deployment workflow, allowing users to train and deploy their own neural networks directly onto the device. With UART and I²C interfaces, it integrates seamlessly with open-source hardware platforms such as Arduino, micro:bit, ESP32, and Raspberry Pi, making it ideal for AI education, STEAM learning, and interactive projects.

In this project, you’ll learn how to connect HUSKYLENS 2 with an Arduino UNO and use gesture recognition to play the classic Angry Birds game.

By detecting your hand movements, HUSKYLENS 2 transforms your gestures into a virtual slingshot — pull back to aim, open your hand to launch — creating a fun and intuitive experience that blends AI vision with real-time gameplay.

Getting Started

Hardware

To follow along with this guide, you’ll need the following:

DFRduino UNO R3 with IO Expansion Shield and USB Cable A-B: A fully Arduino-compatible microcontroller board designed for education and prototyping. It serves as the main controller for sensor data processing and interactive projects, offering seamless compatibility with the Arduino IDE and a wide range of open-source hardware.

HUSKYLENS 2 AI Camera Vision Sensor: A compact and easy-to-play AI vision sensor powered by the Kendryte K230 processor (6 TOPS). It features 20+ preloaded AI models and supports custom model deployment, with UART/I²C connectivity for seamless integration with Arduino, ESP32, and Raspberry Pi.

Software

Arduino IDE 2.3.6 : It is a versatile, beginner-friendly, open-source electronics prototyping platform. With just a bit of learning, you can quickly start developing your own projects.

Use HUSKYLNES 2 to learn gestures

In this section, you’ll learn how to use HUSKYLENS 2 to detect hands, identify keypoints, and recognize specific gestures.

Step 1. Power on HUSKYLENS 2, and select the Hand Recognition function from the main menu. When a hand appears in view, HUSKYLENS 2 automatically detects it — white boxes outline the detected hands, and 21 keypoints on each hand are marked with small white dots.

Step 2. Align your hand (e.g., grab or release gesture) with the center cross on the screen and hold the gesture steady. Press and hold the A button to capture more than 10 learning frames, then release it to proceed to the next gesture. For better results, slowly rotate your wrist and slightly adjust the hand angle during the learning process to improve recognition accuracy.

Step 3. Once learned, HUSKYLENS 2 will highlight the recognized gesture with colored boxes, showing the gesture name, ID, and confidence score on the screen.

Arduino Preparation

Step 1. Download and install the latest version of Arduino IDE according to your operating system

Step 2. Launch the Arduino application

Step 3. Follow the wiring diagram to connect the DFRobot_HuskylensV2 to the power board (a power board is included with HUSKYLENS V2 to ensure more stable operation), and then connect the Arduino UNO to your computer using a USB Cable A-B for Arduino Uno.

Step 4. Select your board and port
Board:
Navigate to Tools > Board > Arduino AVR Boards and select "Arduino Uno".

Port:

Navigate to Tools > Port and select the correct serial port for your board. Ensure the COM number is accurate.

Step 5. Library Installation

To use HUSKYLENS 2 with Arduino, install the DFRobot_HuskylensV2 library from the official DFRobot GitHub repository: DFRobot_HuskylensV2 Library.

1. In the Arduino IDE, go to Sketch → Include Library → Add .ZIP Library...
2. Select the downloaded .zip file to import it.
3. Alternatively, follow the official Arduino tutorial on Installing Libraries (.zip).

Upload the Arduino Code

In this project, the Arduino UNO collects gesture recognition data from HUSKYLENS 2 and sends it to the PC via serial communication.

Step 1. Open or paste the code from Huskylens_angry_birds_game.ino.Click the Upload (right arrow) button. Open the Serial Monitor and set baud rate to 115200.

If you see outputs like:

Then your HUSKYLENS 2 and Arduino UNO are communicating correctly!

Run the Python Game Script

Once Arduino starts sending gesture data, we can use a Python script to control the Angry Birds game.

For full source code access and downloads, please visit the project repository:

🔗 HuskyLens2 Angry Birds Game — GitHub

Run the Game

1. Keep the Arduino UNO connected to your computer.

2. Confirm the serial port (e.g. COM3, COM14).

3. Open a terminal in the project folder and run:

py main_uno.py

The Angry Birds game window will appear.

Structure Printing

The HuskyLens 2 Universal Mount is a 3D-printed holder designed to securely attach the HuskyLens AI Vision Sensor to a computer screen. It supports structures with a thickness of 5 mm – 24 mm and is compatible with both the HuskyLens 1 and HuskyLens 2, as well as other modules with the same mounting hole spacing.

You can download and print the 3D files here to assemble the complete mount:

Now you can control the game through hand gestures and enjoy an interactive experience — pull, aim, and launch just like in the real world!

And this is just the beginning — the HuskyLens 2 AI Vision Sensor offers many more powerful AI features beyond gesture recognition. Let’s explore them together!