1x Micro USB or USB C cable to connect to the computer
Setup instructions
Download Anki Cozmo App, then power on your robot. Connect the app to the robot using Bluetooth, then connect your phone to your computer using the USB Cable.
Make sure to install python 3.9, we could not get the code to work with newer versions of python.
Download all of the appropriate imports using "pip install...".
Connect your phone to the robot using bluetooth.
Place a small piece of tape on a flat surface, then place your robot on top of the this tape.
How to use Our Code
First, change the file path in ImageGathering to an appropriate path for you.
Then, run ImageGathering. Wait for the robot to complete a 360 rotation. The first picture from this process will be your "home" picture and orientation.
Pick up and rotate the robot, place it back down onto the piece of tape to minimize non-rotational movement, which will create unecessary error.
Run SlidingWindow to localize the robot, he will face in the home direction and celebrate with a song. At this point you can read the time spent localizing and read the graph.
As long as the environment remains reasonably unchanged, you may kidnap the robot as many more times as you'd like.
Improvements
Image downscaling has improved speed of localization from the original code by approximately 60s, bringing the average down from around 105 seconds to 45 seconds. This is a 32% improvement.
In the worst case of the original code, localization took about 5 minutes. The worst case we have observed since completing changes is just over 1 minute.
When running the original code, the robot would tilt his head up while collecting pictures, but not while localizing. We made it so he tilts his head while localizing, so there is less variance between images taken from the same angle.
Threshold modification to increase certainty from 40% to 80% has appeared to increase accuracy.
The robot now sings "Ode to Joy" upon localizing
The robot now displays its home image on its OLED display upon localization
For Future Groups
We are currently downsizing images to 2/3 of their original size, and our threshold for certainty is set to 80%. In the future, these numbers could be changed around to try to find an optimal combination.
Image hashing has the potential to resize images with superior efficiency
Before we began downsizing images, we saw success in the speed of localization by splicing the arrays which represented the images so that only the top half of each image was compared. This did come with accuracy drawbacks, although we forgot to experiment with the confidence threshold before moving on from this approach. A future group could try only commiting half of the images to the array, and then adjusting the confidence threshold to adjust for the increased susceptibility to error.
While collecting images, the robot typically moves a couple centimeters off of its original point. Although rotate is in the Cozmo library, a group could add some code after the robot is done taking pictures to put it closer to its start point. This may increase accuracy and allow it to obtain higher certainity faster, improving speed as well. This group would also have to be careful to place the robot back down exactly where it began though, or else the improved accuracy on rotation would not actually reduce error.
Changing the number of photos taken in collection could improve localization, although it would come with a drawback on time to collect photos. To make the entire process run faster, this number could be experimented with.
Image annotation in the Cozmo library has the capacity to downsize images as well, this approach could yield faster downsizing and additional resampling. The comparisons may be simpler and more accurate as well. It would also remove the need to change image type.
Conclusions
Downscaling images is very effective in improving the speed of image comparisons.
Changing the certainty threshold seems necessary if you implement downscaling.