Scout Robot - 3D Vision, SLAM, NLP, Neural Nets

Scout is a robot built to test 3D Vision, SLAM, NLP, neural nets, mapping, pathfinding, etc.  I am using this bot to test features meant for another and larger bot of mine (Ava v2).  My goal is to perfect the ability to move intelligently around my house from any point to any other point accessible to the robot.  I also intend to perfect the 3D perception system and a new spatial 3D memory for everything the bot sees.  This bot fuses data from multiple sensors and neural nets to perform the various functions.  This bot is controlled via voice remote, web page, or game controller.  This bot also has an autonomous mode.


Machine Learning Models

I incorporated the following off-the-shelf neural nets into the brain of this robot.  Most of these models are running in python on a separate laptop with a graphics card,  and accessed through a custom built flask api.


Verbal Models (mostly Transformer Models)

NLP DialoGPT Model (Microsoft)

NLP Text Generation Model

NLP Sentiment Detection Model

NLP Masking Model (Transformer)

NLP Question Answering Model (Transformer)

NLP Entity Recognition Model


Vision Models

YOLO v3 DarkNet Model

AlexNet Model

Gender/Age Detection Model

Face Detection Model

Emotion Detection Model


Significant Libraries & Algorithms

I incorporated the following libraries and algorithms into the software.  This list is only the major ones.

Spacy NLP Library

Open CV Vision - Various algos for color, shape detection, etc.

A Star Pathfinding (in Python)

2D Occupancy Grid Map (in Python)

3D Memory System - (in Python)

Various custom built NLP Algos

Various Graph Algos

Fast Fourier Transforms (for audio spectrum analyzer)


Data Sources & APIs

This bot learns on its own any time any new word is encountered.  For this, I use the following data sources.

Word/Thesaurus API

Wikipedia Text and API

RDF Triple Sources (Dbpedia and others on Linked Open Data Web)

Wolfram Alpha API


GeoNames database

Some custom built SQL Server databases

Weather API

1 MeLE Fanless Mini PC Stick Intel Celeron J4125 8G/128G Windows SBC
1 Orbbec Astra S Image and Depth Camera
1 Intel T265 Tracking Camera (for Pose and SLAM)
1 CMPS10 Tilt Compensated Compass
9 SRF-04 Sonars
2 GM 37 12V Encoder Motors
2 Lasers
2 Neopixel Jewels
1 4-Port Powered USB Hub
1 Motor Controller
1 Servo Controller Board
1 Arduino Mega ADK
1 5in LED Screen
1 USB Speaker
2 Servos
1 4S 4.5Amp Hour LiPo Battery
1 2in Lynxmotion Track and Sprocket Set
All Rights