Introduction

This is Ava.  This is my 3rd bot and is the product of 3 years of work.  Ava combines advanced software, hardware, 3D sensors, machine learning models, and 22-degree of freedom movement to deliver what I believe is my smartest, most capable, and hopefully best looking bot yet. 

This bot uses visual SLAM to map and locate itself.  I plan on using this bot to demonstrate what is possible with state of the art natural language and vision processing using a combination of a custom neural net based architecture and advanced "transformer" based machine leaning models.

A Few Pictures…

Advanced Neural Net Architecture

Conceptually, Ava's brain is a neural net of neural nets.  Each node in the high-level brain can accept inputs, have custom activation rules, and send outputs to other nodes.  Every node has a name and one or more aliases.  Any node can also be autonomous.  Internally each node can have its own internal neural net, fuzzy logic rules, verbal "English Paragraph" script, or procedural code.

Natural Language Orchestration

People can talk to the bot using language.  Because every node in the brain has a name, a person can talk to any node and get, set, increase, decrease, etc. its value.  In the same way, nodes can talk to each other...using natural language.  This allows complex operations (like coordinated movements) to be performed using simple sentences and paragraphs.

Self-Learning 

Ava is self-learning.  It builds knowledge graphs from linked open data sources on the web (such as dbpedia, wikipedia, concept-net etc), databases, APIs, etc.  Ava integrates all the data for any given topic into a single graph.  Ava can build a graph around people, places, users, or any given word or topic.

Natural Language Question Answering & Comprehension

Based on the knowledge graphs, Ava can then answer natural language questions.  Ava can also answer questions based on text sources using a "transformer" BERT-based model for question answering.  Ava can also compare and contrast topics..."Compare hydrogen and helium."

Machine Learning Models

Currently, this bot uses several off-the shelf machine learning models (from the Intel Model Zoo or HuggingFace) to do object recognition (like Yolov3) and other visual functions as well as many natural language-based functions.  Over time, more models will be added.  Some of these models run on the neural compute stick and other run on the LattePanda.