Scout is a robot built to test 3D Vision, SLAM, NLP, neural nets, mapping, pathfinding, etc. I am using this bot to test features meant for another and larger bot of mine (Ava v2). My goal is to perfect the ability to move intelligently around my house from any point to any other point accessible to the robot. I also intend to perfect the 3D perception system and a new spatial 3D memory for everything the bot sees. This bot fuses data from multiple sensors and neural nets to perform the various functions. This bot is controlled via voice remote, web page, or game controller. This bot also has an autonomous mode.
Machine Learning Models
I incorporated the following off-the-shelf neural nets into the brain of this robot. Most of these models are running in python on a separate laptop with a graphics card, and accessed through a custom built flask api.
Verbal Models (mostly Transformer Models)
NLP DialoGPT Model (Microsoft)
NLP Text Generation Model
NLP Sentiment Detection Model
NLP Masking Model (Transformer)
NLP Question Answering Model (Transformer)
NLP Entity Recognition Model
Vision Models
YOLO v3 DarkNet Model
AlexNet Model
Gender/Age Detection Model
Face Detection Model
Emotion Detection Model
Significant Libraries & Algorithms
I incorporated the following libraries and algorithms into the software. This list is only the major ones.
Spacy NLP Library
Open CV Vision - Various algos for color, shape detection, etc.
A Star Pathfinding (in Python)
2D Occupancy Grid Map (in Python)
3D Memory System - (in Python)
Various custom built NLP Algos
Various Graph Algos
Fast Fourier Transforms (for audio spectrum analyzer)
Data Sources & APIs
This bot learns on its own any time any new word is encountered. For this, I use the following data sources.
Word/Thesaurus API
Wikipedia Text and API
RDF Triple Sources (Dbpedia and others on Linked Open Data Web)
Wolfram Alpha API
ConceptNet
GeoNames database
Some custom built SQL Server databases
Weather API
