icon

A Dancing Robot That Listens to Music and Moves to the Notes

I like making music as well as silly robots, so I figured it only made sense to have those worlds collide. The idea is that the robot listens to the different frequencies of whatever song is played and moves based on whatever notes it hears, so it'll dance differently for any given song. I have different maneuvers written into the code based on the given note, but I've included 3 octave ranges and noted which frequencies correlate to which note, so you can very easily setup your own dance moves if you're following along!

HARDWARE LIST
1 Devastator Tank Robot
1 Romeo BLE
1 Arduino Compatible Microphone
1 Proximity Sensor
projectImage
STEP 1
Step one is assembly! I've included the schematics for the electronics in this step. This is a simplified version that doesn't include the tank robot wiring (also depicted) in order to make it more readable. There are only a couple of extra parts to plug in, so we can keep the wiring nice and simple with no need for a breadboard. Conveniently, it's recommended for microphones like these to use 3.3v, whereas the proximity sensor uses the fully 5v. For the proximity sensor, plug the VCC into 5v, Trig into pin 11, Echo into pin 12, and GND into GND. For the microphone, plug VCC into the 3.3v, GND into GND, and OUT into A0. Important note: it does seem that the setup works best with out plugged into A0, but you may run into issues uploading your code from the Arduino IDE while it's plugged in. If that occurs, unplug from A0, upload your code, then plug back into A0. If you're using the same devastator robot I used, I would strongly suggest getting the code fully up and running before putting on the top and wheel attachments, because it's kind of inconvenient to get the Arduino plugged back into the computer once it’s fully assembled.
projectImage
STEP 2
Before we get into making the robot dance, let's make sure it doesn't dance its way into walls. That's where the proximity sensor comes into play. While the robot is moving around the room, it'll also be on the lookout for obstacles and will avoid them accordingly. I have just one forward facing proximity sensor, so the code is very simple for obstacle avoidance, but it would be very easy to add a little more vision to the robot by adding a servo to the sensor so it can look around and/or a second proximity sensor at the back of the robot. For a setup like mine, just mount the proximity sensor to the front of the robot. The code for the project is included in this step.
CODE
//Got some of the code relating to getting frequency intensities with the FFT library https://github.com/TechRandom/Arduino-Audio-Visualizer/blob/master/FFT_Visualizer.ino
//And some code for the robot basics and avoidance here https://github.com/Arduinolibrary/DFRobot_Devastator_Tank_Mobile_Platform_Metal_motor_version


#include <arduinoFFT.h>
#include <DFMobile.h>
DFMobile Robot (4,5,7,6);//initiate to enable DFMobile motion 


#define SAMPLES 64        // Must be a power of 2
#define MIC_IN A0         // Use A0 for mic input
#define xres 8            // Total number of  columns in the display
#define yres 8            // Total number of  rows in the display

double vReal[SAMPLES];
double vImag[SAMPLES];

int Intensity[xres] = { }; // initialize Frequency Intensity to zero
int Displacement = 1;

int trigPin = 11;    // Trigger
int echoPin = 12;    // Echo
long duration, cm, inches;
double distance;//prarmeters of infrared distance measurement

arduinoFFT FFT = arduinoFFT();  // Create FFT object
long nextBeat;
boolean started=false;
double highestIntensity=0;
int currentFrequency;

void setup() {
  pinMode(MIC_IN, INPUT);
  Serial.begin(115200);         //Initialize Serial
  delay(3000);                  // power-up safety delay
}

//Note      Frequency (Hz)  Wavelength (cm)
//C4        261.63          131.87
// C#4/Db4  277.18          124.47
//D4        293.66          117.48
// D#4/Eb4  311.13          110.89
//E4        329.63          104.66
//F4        349.23          98.79
// F#4/Gb4  369.99          93.24
//G4        392.00          88.01
// G#4/Ab4  415.30          83.07
//A4        440.00          78.41
// A#4/Bb4  466.16          74.01
//B4        493.88          69.85
//C5        523.25          65.93

//I'm using approximate frequencies for each note
//There's probably a better way to do this but whatever it works
void dance(){

    Serial.print("Current Frequency: ");
    Serial.println(highestIntensity);

    if((highestIntensity>62 && highestIntensity<67) || (highestIntensity>=127 && highestIntensity<134) || (highestIntensity>=254 && highestIntensity<269))  //C
    {      
      Robot.Speed(-200,-200);//back off 
    }
    else if((highestIntensity>=67 && highestIntensity<71) || (highestIntensity>=134 && highestIntensity<142) || (highestIntensity>=269 && highestIntensity<285))  //C#
    {      
      Robot.Speed(-200,200);//turn left
    } else if((highestIntensity>=71 && highestIntensity<74) || (highestIntensity>=142 && highestIntensity<151) || (highestIntensity>=285 && highestIntensity<302))  //D
    {
      Robot.Speed(200,200);//move forward
    } else if((highestIntensity>=74 && highestIntensity<80) || (highestIntensity>=151 && highestIntensity<160) || (highestIntensity>=302 && highestIntensity<320))  //D#
    {
      Robot.Speed(-200,-200);//back off 
      delay(50);
      Robot.Speed(-200,200);//turn left
      delay(200);
    } else if((highestIntensity>=80 && highestIntensity<85) || (highestIntensity>=160 && highestIntensity<169) || (highestIntensity>=320 && highestIntensity<338))  //E
    {      
      Robot.Speed(200,-200);//turn right
    } else if((highestIntensity>=85 && highestIntensity<90) || (highestIntensity>=169 && highestIntensity<179) || (highestIntensity>=338 && highestIntensity<359))  //F
    {      
      Robot.Speed(-50,-50);//scoot backward
      delay(50);
      Robot.Speed(-50,-50);//scoot backward
      delay(50);
      Robot.Speed(-50,-50);//scoot backward
      delay(50);
      Robot.Speed(-50,-50);//scoot backward
      delay(50);
    } else if((highestIntensity>=90 && highestIntensity<95) || (highestIntensity>=179 && highestIntensity<189) || (highestIntensity>=359 && highestIntensity<381))  //F#
    {
      Robot.Speed(-200,-200);//back off 
      delay(50);
      Robot.Speed(200,-200);//turn right
      delay(200);
    } else if((highestIntensity>=95 && highestIntensity<101) || (highestIntensity>=189 && highestIntensity<201) || (highestIntensity>=381 && highestIntensity<403))  //G
    {      
      Robot.Speed(50,50);//scoot forward
      delay(50);
      Robot.Speed(50,50);//scoot forward
      delay(50);
      Robot.Speed(50,50);//scoot forward
      delay(50);
      Robot.Speed(50,50);//scoot forward
      delay(50);
    } else if((highestIntensity>=101 && highestIntensity<107) || (highestIntensity>=201 && highestIntensity<214) || (highestIntensity>=403 && highestIntensity<427))  //G#
    {
      Robot.Speed(-150,50);//angle left      
    } else if((highestIntensity>=107 && highestIntensity<113) || (highestIntensity>=214 && highestIntensity<227) || (highestIntensity>=427 && highestIntensity<453))  //A
    {      
      Robot.Speed(600,-600);//spin right
    } else if((highestIntensity>=113 && highestIntensity<120) || (highestIntensity>=227 && highestIntensity<237) || (highestIntensity>=453 && highestIntensity<479))  //A#
    {
      Robot.Speed(-600,600);//spin left
    } else if((highestIntensity>=120 && highestIntensity<127) || (highestIntensity>=237 && highestIntensity<254) || (highestIntensity>=479 && highestIntensity<515))  //B
    {      
      Robot.Speed(50,-150);//angle right
    }

}

//This came from FFT_Visualizer.ino and some of what I left in is unnecessary to this project, but it's just that much more info available to work with
void getSamples(){
  for(int i = 0; i < SAMPLES; i++){
    vReal[i] = analogRead(MIC_IN);
    //Uncomment only for initial testing - you'll get a lot of values
    //Serial.println(vReal[i]);
    vImag[i] = 0;
    if(vReal[i]>highestIntensity && vReal[i]>230 && vReal[i]<550)
    {
          highestIntensity=vReal[i];
          currentFrequency=(i/Displacement)-2;
          
          Serial.print("Current Frequency: ");
          Serial.println(highestIntensity);
        }
  }
  FFT.Windowing(vReal, SAMPLES, FFT_WIN_TYP_HAMMING, FFT_FORWARD);
  FFT.Compute(vReal, vImag, SAMPLES, FFT_FORWARD);
  FFT.ComplexToMagnitude(vReal, vImag, SAMPLES);

  //Update Intensity Array
  for(int i = 2; i < (xres*Displacement)+2; i+=Displacement){
    vReal[i] = constrain(vReal[i],0 ,2047);            // set max value for input data
    vReal[i] = map(vReal[i], 0, 2047, 0, yres);        // map data to fit our display

    Intensity[(i/Displacement)-2] --;                      // Decrease displayed value
    if (vReal[i] > Intensity[(i/Displacement)-2])          // Match displayed value to measured value
      {
        Intensity[(i/Displacement)-2] = vReal[i];
      }
  }
}


void checkForObstacles(){
  digitalWrite(trigPin, LOW);
  delayMicroseconds(5);
  digitalWrite(trigPin, HIGH);
  delayMicroseconds(10);
  digitalWrite(trigPin, LOW);
 
  // Read the signal from the sensor: a HIGH pulse whose
  // duration is the time (in microseconds) from the sending
  // of the ping to the reception of its echo off of an object.
  pinMode(echoPin, INPUT);
  duration = pulseIn(echoPin, HIGH);
 
  // Convert the time into a distance
  cm = (duration/2) / 29.1;     // Divide by 29.1 or multiply by 0.0343
  inches = (duration/2) / 74;   // Divide by 74 or multiply by 0.0135
  
  Serial.print(inches);
  Serial.print("in, ");
  Serial.print(cm);
  Serial.print("cm");
  Serial.println();

  distance = cm;

  //If the robot moves too close to something, make it face somewhere else so it's not just bumping into stuff
  if(distance < 25){
    
  Serial.println("OBSTACLE DETECTED! REPOSITIONING!");
    Robot.Speed (0,0);  // stop
      Robot.Speed(-200,-200);//back off 
      delay(50);
      Robot.Speed(200,-200);//turn right
      delay(200);
    Robot.Speed (100,100);  //forward
  }
}


void loop() {

  //Only start dancing once we detect something other than the resting frequency of the room
  if(!started && highestIntensity>0 && (highestIntensity>350 || highestIntensity<339)){
    started=true;  
    nextBeat= millis()+(2000L);
    Serial.print("Current Frequency1: ");
    Serial.println(highestIntensity);
  }
  else{
    //Dance on the beats only
    if(millis()>=nextBeat){
      dance();
      highestIntensity=0;
      nextBeat= millis()+(2000L); //This is equivalent to a whole note at 120BPM
    }
  }
  getSamples();
}
projectImage
STEP 3
The goal is to figure out what note is most prominent during any given beat. For the duration of a whole note at 120BPM, we listen for the frequency with the highest intensity. We then figure out what note correlates to that frequency, and have the robot move accordingly. The way I tackled the frequency to note logic is more tedious than I'd have preferred, with 3 octaves worth of if statements, but it accomplishes the goal of determining what the robot needs to do based on the current frequency. As part of the logic, I found the frequency of the room without music playing to keep the robot from dancing if it hears something different. There are a lot of subtle factors that go into the microphone readings, so it's likely you'll get slightly different results. As such, it likely makes sense to see what readings your microphone picks up for your room as a baseline, and it'd be interesting to see what kind of readings you get with music before getting rolling with the fully mobile tank robot. It's pretty nice to be able to just mess with the audio testing without all the extra stuff involved in a tank robot, hence the image of the Elegoo Uno with just the microphone setup. Another important note is that the robot isn't particularly quiet itself, so the microphone may pick up noises from itself and dance based on those. As such, I added a stand on top of the robot so that I could have the mic up and away from the wheels.
projectImage
STEP 4
The robot should now be well equipped to dance to any song. It'll also more or less do the same dance to any given song, which is pretty sweet. If it's not reacting to the notes of the song well enough, you may need to increase the volume or move the music closer to the robot, because it's prioritizing whatever note it hears most and may be hearing its own wheels moving. The code can be easily modified to add in more or different dance moves, so enjoy taking a stab at your own dancing robot if that's your jam and, if that's not your cup of tea, thanks for reading along and hopefully you enjoyed anyway!
License
All Rights
Reserved
licensBg
0