Greetings everyone, and welcome to my article tutorial. Today, I'll guide you through the process of creating an AI Pin.
Project Overview:
This project is built using the Seeed Studio XIAO ESP32S3 Sense board. It connects with a Telegram bot to receive commands and interact with the user. Once a query is sent, the device captures an image and forwards it along with the question to the Gemini AI API. The response is then delivered back to the user directly on Telegram. This is a great project for students, as it combines IoT and AI concepts, making it perfect for hands-on learning and even as a college project.
From a technical perspective, the ESP32S3 Sense is configured to use its onboard camera for capturing images. It connects to Wi-Fi, communicates with Telegram servers, and handles secure HTTPS requests to Google’s Gemini API. Base64 encoding is used to transfer image data along with the text query. The code manages camera frames, parses API responses using ArduinoJson, and sends AI-generated replies to Telegram. This setup demonstrates practical integration of hardware, networking, and AI services in a compact IoT system.
Before beginning, a huge shoutout to JLCMC for sponsoring.
Now, let's get started with our project!
Supplies
Electronic Components Required:
-Seeed Studio XIAO ESP32S3 SENSE
-3.7V LI-PO Battery (400 mAh)
-Slide Switch
-Cardboard / Acrylic Sheet / Sun Board, etc.
Additional Tools:
-Hot Glue
-Cutter
-Soldering Iron
Software:
-Arduino IDE
For Arduino IDE Setup Follow these steps:
-Open Arduino IDE on your computer.
-Go to File > Preferences.
-In the Additional Board Manager URLs field, paste this link:
https://raw.githubusercontent.com/espressif/arduino-esp32/gh-pages/package_esp32_index.json
-Then click OK.
-Next, go to Tools > Board > Board Manager.
-Search for ESP32, then click Install.
Now install the required library:
-Open Sketch > Include Library > Manage Libraries.
-Search for UniversalTelegramBot.h.
-Install it.
Create a Telegram Bot:
-Open the Telegram app on your phone.
-Search for @BotFather (the official Telegram bot creator).
-Start a chat and type /newbot.
-Give your bot a name and a unique username.
-BotFather will generate a token for your bot. (Copy and save this token; we will use it in the Arduino code.)
About this step:
we’ll write a code that allows our Telegram bot to take a command from the user and reply with a simple pre-set message
#include <WiFi.h>
#include <WiFiClientSecure.h>
#include <UniversalTelegramBot.h>
// ============ Replace these with your details ============
const char* ssid = "WIFI_NAME";
const char* password = "WIFI_PASSWORD";
const char* botToken = "TELEGRAM_BOT_TOKEN"; // Your Telegram Bot Token
// ========================================================
// Setup secure WiFi client for HTTPS
WiFiClientSecure netClient;
UniversalTelegramBot bot(botToken, netClient);
unsigned long lastCheckTime = 0;
const unsigned long checkInterval = 1000;
void setup() {
Serial.begin(115200);
// Wi-Fi connect
WiFi.begin(ssid, password);
Serial.print("Connecting to WiFi");
while (WiFi.status() != WL_CONNECTED) {
delay(500);
Serial.print(".");
}
Serial.println("\nConnected.");
netClient.setInsecure();
Serial.println("Telegram Bot started.");
}
void loop() {
// Periodically check for new messages
if (millis() - lastCheckTime >= checkInterval) {
checkTelegramBot();
lastCheckTime = millis();
}
}
void checkTelegramBot() {
int numNewMessages = bot.getUpdates(bot.last_message_received + 1);
while (numNewMessages) {
for (int i = 0; i < numNewMessages; i++) {
String chat_id = String(bot.messages[i].chat_id);
String text = bot.messages[i].text;
Serial.print("Received: "); Serial.println(text);
if (text == "/ask") {
String response = "Hello, bro!";
bot.sendMessage(chat_id, response, "");
Serial.println("Sent reply: Hello, bro!");
}
}
numNewMessages = bot.getUpdates(bot.last_message_received + 1);
}
}Connect the board to your computer using a USB-C cable. With the Arduino IDE open and your code ready, ensure you have selected the correct board and port by navigating to the Tools menu. Then, click the Upload button (the right-arrow icon) to begin compiling and flashing the sketch to the XIAO ESP32S3.
JLCMC is your one-stop shop for all electronic manufacturing needs, offering an extensive catalog of nearly 600,000 SKUs that cover hardware, mechanical, electronic, and automation components. Their commitment to guaranteeing genuine products, rapid shipping (with most in-stock items dispatched within 24 hours), and competitive pricing truly sets them apart. In addition, their exceptional customer service ensures you always get exactly what you need to bring your projects to life.
They have everything you need for your next project:
-Custom Linear Guide Shafts: Precision-engineered for applications like 3D printing, CNC machines, and industrial automation.
-Aluminum Profiles: Versatile, durable framing solutions—perfect for machine enclosures, workstations, and custom assemblies.
To show their support for our community, JLCMC is offering an exclusive $70 discount coupon. This is the perfect opportunity to save on high-quality components for your next project. Don’t miss out—visit https://jlcmc.com/?from=RBL to explore their amazing range of products and grab your discount coupon today!
About this step:
We’ll upgrade that same code so the bot can use an AI model to reply with intelligent responses.
Follow these steps to create the Gemini API key:
-Open your browser and go to https://aistudio.google.com/
-Click the Create API key button. You can create a key in a new or existing Google Cloud project.
-The system will generate a new API key. Click the Copy button to save it. Paste the key into the designated placeholder in the code.
#include <WiFi.h>
#include <WiFiClientSecure.h>
#include <UniversalTelegramBot.h>
#include <ArduinoJson.h>
// ==== CONFIG ====
const char* ssid = "WIFI_NAME";
const char* password = "WIFI_PASSWORD";
const char* botToken = "TELEGRAM_BOT_TOKEN"; // Your Telegram Bot Token
const char* GEMINI_API_KEY = "GEMINI_API_KEY"; // Gemini API Key
// ================
// Telegram bot client
WiFiClientSecure netClient;
UniversalTelegramBot bot(botToken, netClient);
unsigned long lastCheckTime = 0;
const unsigned long checkInterval = 1000;
bool waitingForQuery = false;
String callGeminiAPI(String userPrompt) {
WiFiClientSecure client;
client.setInsecure();
if (!client.connect("generativelanguage.googleapis.com", 443)) {
Serial.println("❌ Gemini API connection failed");
return "Error: Could not connect to Gemini API";
}
// Create JSON payload
String payload = "{";
payload += "\"contents\":[{\"role\":\"user\",\"parts\":[{\"text\":\"" + userPrompt + "\"}]}],";
payload += "\"generationConfig\":{\"thinkingConfig\":{\"thinkingBudget\":-1}},";
payload += "\"tools\":[{\"googleSearch\":{}}]";
payload += "}";
// Send POST request
String url = "/v1beta/models/gemini-2.5-flash:generateContent?key=" + String(GEMINI_API_KEY);
client.println("POST " + url + " HTTP/1.1");
client.println("Host: generativelanguage.googleapis.com");
client.println("Content-Type: application/json");
client.println("Content-Length: " + String(payload.length()));
client.println("Connection: close");
client.println();
client.print(payload);
while (client.connected() && !client.available()) delay(10);
String response;
while (client.available()) {
response += client.readString();
}
Serial.println("📩 Raw Gemini Response:");
Serial.println(response);
int jsonStart = response.indexOf("\r\n\r\n");
if (jsonStart == -1) return "Error: No JSON found";
String chunkedBody = response.substring(jsonStart + 4);
String jsonBody;
int pos = 0;
while (pos < chunkedBody.length()) {
int crlf = chunkedBody.indexOf("\r\n", pos);
if (crlf == -1) break;
String sizeStr = chunkedBody.substring(pos, crlf);
int chunkSize = strtol(sizeStr.c_str(), NULL, 16);
if (chunkSize == 0) break;
jsonBody += chunkedBody.substring(crlf + 2, crlf + 2 + chunkSize);
pos = crlf + 2 + chunkSize + 2;
}
StaticJsonDocument<4096> doc;
DeserializationError error = deserializeJson(doc, jsonBody);
if (error) {
Serial.println("❌ JSON parse failed: " + String(error.c_str()));
return "Error: Could not parse JSON";
}
if (doc["candidates"][0]["content"]["parts"][0]["text"].is<String>()) {
return doc["candidates"][0]["content"]["parts"][0]["text"].as<String>();
}
return "Error: No text found";
}
void setup() {
Serial.begin(115200);
WiFi.begin(ssid, password);
Serial.print("Connecting to WiFi");
while (WiFi.status() != WL_CONNECTED) {
delay(500);
Serial.print(".");
}
Serial.println("\n✅ WiFi connected!");
netClient.setInsecure();
}
void loop() {
if (millis() - lastCheckTime >= checkInterval) {
checkTelegramBot();
lastCheckTime = millis();
}
}
void checkTelegramBot() {
int numNewMessages = bot.getUpdates(bot.last_message_received + 1);
while (numNewMessages) {
for (int i = 0; i < numNewMessages; i++) {
String chat_id = String(bot.messages[i].chat_id);
String text = bot.messages[i].text;
Serial.print("📨 Received: ");
Serial.println(text);
if (text == "/ask") {
bot.sendMessage(chat_id, "Send me your query", "");
waitingForQuery = true; // set state
}
else if (waitingForQuery) {
waitingForQuery = false; // reset state
bot.sendMessage(chat_id, "⏳ Thinking...", "");
String geminiResponse = callGeminiAPI(text);
bot.sendMessage(chat_id, geminiResponse, "");
}
}
numNewMessages = bot.getUpdates(bot.last_message_received + 1);
}
}Copy and paste this code in the Arduino IDE, and then click the Upload button.
In this step, our goal is to capture an image when the bot receives a command, then convert it into Base64 format, and finally, send back the first 50 characters as confirmation.
Steps to enable the PSRAM:
Before uploading the code to your XIAO ESP32S3 Sense board, you need to enable PSRAM (external memory).
-In the Arduino IDE, go to the Tools menu.
-Find the option “PSRAM”
-Set it to “Enabled”
✅ This ensures your XIAO ESP32S3 Sense board has enough memory to handle Image Capture & Gemini API responses.
#include "esp_camera.h"
#include <WiFi.h>
#include "base64.h"
#include <WiFiClientSecure.h>
#include <UniversalTelegramBot.h>
// Pin definitions for XIAO ESP32S3 Sense
#define PWDN_GPIO_NUM -1
#define RESET_GPIO_NUM -1
#define XCLK_GPIO_NUM 10
#define SIOD_GPIO_NUM 40
#define SIOC_GPIO_NUM 39
#define Y9_GPIO_NUM 48
#define Y8_GPIO_NUM 11
#define Y7_GPIO_NUM 12
#define Y6_GPIO_NUM 14
#define Y5_GPIO_NUM 16
#define Y4_GPIO_NUM 18
#define Y3_GPIO_NUM 17
#define Y2_GPIO_NUM 15
#define VSYNC_GPIO_NUM 38
#define HREF_GPIO_NUM 47
#define PCLK_GPIO_NUM 13
// ============ Replace these with your details ============
const char* ssid = "WIFI_NAME";
const char* password = "WIFI_PASSWORD";
const char* botToken = "TELEGRAM_BOT_TOKEN"; // Your Telegram Bot Token
// ========================================================
WiFiClientSecure netClient;
UniversalTelegramBot bot(botToken, netClient);
unsigned long lastCheckTime = 0;
const unsigned long checkInterval = 1000;
String handleCapture() {
camera_fb_t * fb = esp_camera_fb_get();
if(!fb) {
Serial.println("Camera capture failed");
return "Camera capture failed!";
}
// Convert image to base64
String base64String = base64::encode(fb->buf, fb->len);
// Return the frame buffer
esp_camera_fb_return(fb);
// Send base64 string
return (base64String);
}
void setup() {
Serial.begin(115200);
Serial.println();
// Camera configuration
camera_config_t config;
config.ledc_channel = LEDC_CHANNEL_0;
config.ledc_timer = LEDC_TIMER_0;
config.pin_d0 = Y2_GPIO_NUM;
config.pin_d1 = Y3_GPIO_NUM;
config.pin_d2 = Y4_GPIO_NUM;
config.pin_d3 = Y5_GPIO_NUM;
config.pin_d4 = Y6_GPIO_NUM;
config.pin_d5 = Y7_GPIO_NUM;
config.pin_d6 = Y8_GPIO_NUM;
config.pin_d7 = Y9_GPIO_NUM;
config.pin_xclk = XCLK_GPIO_NUM;
config.pin_pclk = PCLK_GPIO_NUM;
config.pin_vsync = VSYNC_GPIO_NUM;
config.pin_href = HREF_GPIO_NUM;
config.pin_sccb_sda = SIOD_GPIO_NUM;
config.pin_sccb_scl = SIOC_GPIO_NUM;
config.pin_pwdn = PWDN_GPIO_NUM;
config.pin_reset = RESET_GPIO_NUM;
config.xclk_freq_hz = 20000000;
config.frame_size = FRAMESIZE_SVGA;
config.pixel_format = PIXFORMAT_JPEG;
config.grab_mode = CAMERA_GRAB_WHEN_EMPTY;
config.fb_location = CAMERA_FB_IN_PSRAM;
config.jpeg_quality = 12;
config.fb_count = 1;
if(psramFound()){
config.jpeg_quality = 10;
config.fb_count = 2;
config.grab_mode = CAMERA_GRAB_LATEST;
} else {
config.frame_size = FRAMESIZE_VGA;
config.fb_location = CAMERA_FB_IN_DRAM;
}
// Initialize camera
esp_err_t err = esp_camera_init(&config);
if (err != ESP_OK) {
Serial.printf("Camera init failed with error 0x%x", err);
return;
}
// Wi-Fi connect
WiFi.begin(ssid, password);
Serial.print("Connecting to WiFi");
while (WiFi.status() != WL_CONNECTED) {
delay(500);
Serial.print(".");
}
Serial.println("\nConnected.");
netClient.setInsecure();
Serial.println("Telegram Bot started.");
}
void checkTelegramBot() {
int numNewMessages = bot.getUpdates(bot.last_message_received + 1);
while (numNewMessages) {
for (int i = 0; i < numNewMessages; i++) {
String chat_id = String(bot.messages[i].chat_id);
String text = bot.messages[i].text;
Serial.print("Received: "); Serial.println(text);
if (text == "/ask") {
String response = handleCapture();
Serial.println(response);
response = response.substring(0, min((int)response.length(), 50));
bot.sendMessage(chat_id, response, "");
Serial.println("Sent reply (first 50 chars of base64)");
}
}
numNewMessages = bot.getUpdates(bot.last_message_received + 1);
}
}
void loop() {
if (millis() - lastCheckTime >= checkInterval) {
checkTelegramBot();
lastCheckTime = millis();
}
}Upload and Test the Code
-Connect your XIAO ESP32S3 board to your computer.
-Click the Upload button in the Arduino IDE to flash the code.
-Once uploaded, open the Serial Monitor (set baud rate to 115200).
-Watch the output to confirm that the XIAO ESP32S3 connects to Wi-Fi and starts the bot.
Now it’s time to combine everything. This new code merges Step 4 and Step 5.
Here’s how it works: The bot first asks for a query. Once the user sends it, the XIAO ESP32S3 captures an image, converts it into Base64, and, along with the user query, sends it to the Gemini API. The API processes both inputs and returns a detailed reply.
ALL CODE CAN BE FOUND IN THIS GITHUB REPOSITORY: https://github.com/ShahbazCoder1/AI-Pin
#include "esp_camera.h"
#include <WiFi.h>
#include "base64.h"
#include <WiFiClientSecure.h>
#include <UniversalTelegramBot.h>
// Pin definitions for XIAO ESP32S3 Sense
#define PWDN_GPIO_NUM -1
#define RESET_GPIO_NUM -1
#define XCLK_GPIO_NUM 10
#define SIOD_GPIO_NUM 40
#define SIOC_GPIO_NUM 39
#define Y9_GPIO_NUM 48
#define Y8_GPIO_NUM 11
#define Y7_GPIO_NUM 12
#define Y6_GPIO_NUM 14
#define Y5_GPIO_NUM 16
#define Y4_GPIO_NUM 18
#define Y3_GPIO_NUM 17
#define Y2_GPIO_NUM 15
#define VSYNC_GPIO_NUM 38
#define HREF_GPIO_NUM 47
#define PCLK_GPIO_NUM 13
// ============ Replace these with your details ============
const char* ssid = "WIFI_NAME";
const char* password = "WIFI_PASSWORD";
const char* botToken = "TELEGRAM_BOT_TOKEN"; // Your Telegram Bot Token
// ========================================================
WiFiClientSecure netClient;
UniversalTelegramBot bot(botToken, netClient);
unsigned long lastCheckTime = 0;
const unsigned long checkInterval = 1000;
String handleCapture() {
camera_fb_t * fb = esp_camera_fb_get();
if(!fb) {
Serial.println("Camera capture failed");
return "Camera capture failed!";
}
// Convert image to base64
String base64String = base64::encode(fb->buf, fb->len);
// Return the frame buffer
esp_camera_fb_return(fb);
// Send base64 string
return (base64String);
}
void setup() {
Serial.begin(115200);
Serial.println();
// Camera configuration
camera_config_t config;
config.ledc_channel = LEDC_CHANNEL_0;
config.ledc_timer = LEDC_TIMER_0;
config.pin_d0 = Y2_GPIO_NUM;
config.pin_d1 = Y3_GPIO_NUM;
config.pin_d2 = Y4_GPIO_NUM;
config.pin_d3 = Y5_GPIO_NUM;
config.pin_d4 = Y6_GPIO_NUM;
config.pin_d5 = Y7_GPIO_NUM;
config.pin_d6 = Y8_GPIO_NUM;
config.pin_d7 = Y9_GPIO_NUM;
config.pin_xclk = XCLK_GPIO_NUM;
config.pin_pclk = PCLK_GPIO_NUM;
config.pin_vsync = VSYNC_GPIO_NUM;
config.pin_href = HREF_GPIO_NUM;
config.pin_sccb_sda = SIOD_GPIO_NUM;
config.pin_sccb_scl = SIOC_GPIO_NUM;
config.pin_pwdn = PWDN_GPIO_NUM;
config.pin_reset = RESET_GPIO_NUM;
config.xclk_freq_hz = 20000000;
config.frame_size = FRAMESIZE_SVGA;
config.pixel_format = PIXFORMAT_JPEG;
config.grab_mode = CAMERA_GRAB_WHEN_EMPTY;
config.fb_location = CAMERA_FB_IN_PSRAM;
config.jpeg_quality = 12;
config.fb_count = 1;
if(psramFound()){
config.jpeg_quality = 10;
config.fb_count = 2;
config.grab_mode = CAMERA_GRAB_LATEST;
} else {
config.frame_size = FRAMESIZE_VGA;
config.fb_location = CAMERA_FB_IN_DRAM;
}
// Initialize camera
esp_err_t err = esp_camera_init(&config);
if (err != ESP_OK) {
Serial.printf("Camera init failed with error 0x%x", err);
return;
}
// Wi-Fi connect
WiFi.begin(ssid, password);
Serial.print("Connecting to WiFi");
while (WiFi.status() != WL_CONNECTED) {
delay(500);
Serial.print(".");
}
Serial.println("\nConnected.");
netClient.setInsecure();
Serial.println("Telegram Bot started.");
}
void checkTelegramBot() {
int numNewMessages = bot.getUpdates(bot.last_message_received + 1);
while (numNewMessages) {
for (int i = 0; i < numNewMessages; i++) {
String chat_id = String(bot.messages[i].chat_id);
String text = bot.messages[i].text;
Serial.print("Received: "); Serial.println(text);
if (text == "/ask") {
String response = handleCapture();
Serial.println(response);
response = response.substring(0, min((int)response.length(), 50));
bot.sendMessage(chat_id, response, "");
Serial.println("Sent reply (first 50 chars of base64)");
}
}
numNewMessages = bot.getUpdates(bot.last_message_received + 1);
}
}
void loop() {
if (millis() - lastCheckTime >= checkInterval) {
checkTelegramBot();
lastCheckTime = millis();
}
}Copy and paste this code in the Arduino IDE and upload it!
See the magic...
To make our AI Pin portable, we’ll need a few additional components: a 3.7V Li-Po battery and a small switch. These will power the XIAO ESP32S3 Sense board.
Connect the Battery with a Switch
-Solder the negative terminal of the battery directly to the GND pin of the XIAO ESP32S3 Sense board.
-Take the positive terminal of the battery and connect it to one side of a switch.
-From the other side of the switch, connect a wire to the 3V3 (or VIN) pin on the board.
Default battery charging support
The XIAO ESP32S3 board includes a built-in power management chip for charging a connected lithium battery via its USB-C port.
Here is how the charging functionality works by default:
-Charging indicator: A red LED on the board serves as the charging indicator.
Power and charging status:
-Board connected to USB without a battery: The red LED will turn on briefly and then turn off after about 30 seconds.
-Battery connected and charging via USB: The red LED will flash to indicate that the battery is currently charging.
-Battery fully charged: When the battery is full, the red LED will turn off.
To construct the case, I will use cardboard, but you can use any material, such as acrylic sheet/sun board, etc., as per your requirement. The PDF file containing the cutout templates is attached below. Please refer to the accompanying images for guidance on assembling the case for this project.
AI PIN CUTOUT.zip 
Congratulations! You’ve successfully built your own AI Pin. A demonstration video of this project can be viewed here: Watch Now
Thank you for your interest in this project. If you have any questions or suggestions for future projects, please leave a comment, and I will do my best to assist you.
For business or promotional inquiries, please contact me via email at Email.
I will continue to update this article with new information. Don’t forget to follow me for updates on new projects and subscribe to my YouTube channel (YouTube: roboattic Lab) for more content. Thank you for your support.
