Anki Cozmo Localization

Project Overview

Gettysburg College Computer Science

CS371 Spring 2023

Authors: Binh Tran, Owen Gilman, Spencer Hagan, Ben Durham

Goals

We set out to program the Anki Cozmo robot to be able to localize itself rotationally using Monte Carlo localization. As such, we should be able to do the following: have the robot familiarize itself with its environment, including noting a specified “home location,” rotate it arbitrarily such that it is no longer facing the home position, and the robot should be able to rotate itself back to the home position after doing some sort of process to figure out where it is.

Materials Needed

• 1 Anki Cozmo robot
• 1 Android phone (we used an Android phone. It may be possible with an iPhone but we cannot comment on that)
• At least one computer with the below software installed on it with a way of connecting to an Android phone
• A cable to connect the phone to the computer, probably micro USB or USB-C

Software Prerequisites

• The Cozmo app
• Python 3.10 or higher
• Visual Studio Code

• Jupyter Notebooks extension

• Python libraries to import:

• Numpy
• openCV
• Math
• random
• Cozmo
• MatPlotLib.pyplot
• Os
• sys

Setup Instructions

• For each computer you would like to be able to actually run programs on the robot from, you must install the android debug bridge and the cozmo SDK
• If Python is not installed already, go to https://www.python.org/downloads/ and download the latest version available, or at least 3.10.
• Download and install VSCode from the link https://code.visualstudio.com/download. We preferred to use VSCode for our project since it has particularly good Jupyter Notebook integrations.

Brief Overview of Our Work

In this project, we write a program to help localize the Anki Cozmo robot rotationally. To do this, we first take many pictures of the robot's environment to help it "get its bearings." The first picture it takes denotes its "home position" Then, after a random rotation, either by external intervention or programmatically, the robot should be able to take pictures to try to figure out where it is. From this information, it should then rotate back to its home position.

We apply Monte Carlo localization in the robot's attempt to find where it is. As such, we start with a randomly generated population of possible rotational positions. Then, based on the robot's movement and the image difference with its current location, it makes a new generation of possible positions, with emphasis on the locations that seem most likely, as determined by the image difference between the robot's current position and each of the possible positions.

Our Code and How to Use It

We have our code available here for direct download as python files such that they can be viewed in the browser:

ImageGathering.py

SlidingWindowLocalize.py

However, we recommend getting the jupyter notebook versions. They can be accessed via:

ImageGathering.ipynb

SlidingWindowLocalize.ipynb

(But these are not in browser friendly format, so we recommend downloading them and opening them in VSCode).

Our code can also be downloaded from this repository or cloned using the following command: git clone https://github.com/bendurham441/cs371-cozmo-fourthhour.git. We suggest that future groups fork this repository to have their own working copy.

Image Gathering

One should first run the ImageGathering.ipynb which creates an images/ directory and populates it with images. Note that you need to update the parent_dir variable in the second code block of this file to reflect the actual file path on your computer. After this, this file can be run from top to bottom as a whole. As stated before, the first image taken becomes the robot's home location, so place the robot according to which direction you want to be the home position.

Sliding Window Localize

After the images/ folder is populated, one can now run the contents of SlidingWindowLocalize.ipynb. Note that this file contains a programmatic random rotation of the robot for testing purposes, but is commented out to allow for manual random rotations of the robot. In its current state, the robot will wait for the user to rotate it, then it will start the localization process.

As the robot continues to iterate, this file will print out histograms to the Jupyter Notebook output showing the robot's beliefs about where it is currently facing, which can be useful in seeing the robot zero in on its actual location. These histograms simply show how many locations in the current population are in each "bin."

Conclusion

We have found this method to be very successful under most conditions, in multiple different environments. However, there have been some (rare) cases where the method fails and rotates back to a position that is not the home position.

Future Steps

It is possible that our approach would not be as successful in other environments. We suggest tuning some of the various parameters including

• the size of the pose population
• the number of pictures taken to get the robot’s bearings
• the distance between pictures taken during the localization process
• The process for selecting a position given updated information after iterations of the localization process

• The bin width of degrees should be used in this calculation
• The probability threshold after which the robot truly believes it knows where it is looking

• How image differences are turned into relative probabilities: We used an approach that multiplies the square differences of the reciprocals by the exponential of the squared differences divided by the sensor variance. Admittedly, we took this from the previous group without fully thinking about it since it seemed to work for our purposes. Maybe this could be made better by trying different approaches which might make the robot’s beliefs converge faster.
• In the sliding window process, the degree increments between each offset image. Currently, we shift it by 10 pixels each time. The endpoints of this process (-90px to 90px) could also be tuned to make our approach more efficient.
• Maybe our approach could be made faster by downscaling images. We did not experiment with this, but presumably, it could speed up the process if this was implemented. We do not know how this would affect the efficacy of our method.