• Panorama stitching
  The error that the previous group was having was that their panorama was not stitching all the images together and was leaving out large chunks. We changed the angle the robot turns before taking a new picture, from 10 degrees to 15. This then decreased the amount of images we needed to stitch together.
• Detection of kidnaping
  Instead of letting the robot perform a random turn to simulate the “kidnap”, we made a new function for the robot to detect being picked up, and after it has been set down, it will run the mcl algorithm.
• MCL algorithm
  The previous group only used the mcl algorithm to predict the location without physically locating the robot. We changed this by fixing the motion and measurement models, as well changing their implementation of the mcl algorithm.
• Add function to detect group with the most particles
  We then added a new function that looks at every group of particles, and whichever group has the most particles, we take the mean of that group as our guess for the location. The size of the group converges noticeably as the robot turns and figures out where it is. (We at first simply used the mode of the particles and the convergence was negligible). The robot stops turning when the size of the group exceeds 25 (this number can be adjusted to affect accuracy. See the section that follows).
• Image comparation
  For comparing the images we changed the code to use the ImageChops.difference function to allow us to get an estimated difference between a slice of the panorama and the latest image from Cozmo. We then used this difference in our measurement model to find the weights of the particles. This comparison gives more accurate results compared to the previous group’s comparison using mean-square error (assuming it was implemented correctly).
• Add function to find the home image within the panorama
  The function allowed us to find the location of the home position within the panorama, thus allowing us to compute the number of degrees that Cozmo needed to turn from his current position to get back home.
• Convergence in histogram
  The histogram implemented by the previous group visualizes the convergence.

What still needs to be fixed, possible improvements, and special behaviors:

• Better prediction
  Cozmo makes good predictions of his location, but this prediction could be better, as it is consistently off by around 10 degrees. This could be improved by finding a better tuning for the diff argument and the bound for the checker (if size > *: break) in the mcl algorithm.
• Panorama Stitching
  The panorama stitching is still a little fidgety as it sometimes doesn’t stitch images if they are too similar or too different. Also an interesting behavior we observed was that the panorama is almost always decent when the robot is set on the ground with distinguishable items in the environment. The panorama is almost always bad when the robot is set on an elevated platform. Also the stiiching process can take a very long time, and in that regard makes the whole process inefficient.
• Waiting time for dislocating robots
  When you dislocate the robot, watch out for the timing. And make sure to not block it camera. Refer to on_robot_picked_up() in take_picture.py for details. The implementation can be easily modified.
• Termination of program
  Our current method to terminate the program is to use control-c in the terminal multiple times. This could be done in a better way, i.e. when the button on the robot is pressed, when it’s lifted in a special way, when the cube is tapped...
• Final Advice
  Making the panorama can in principle give a more accurate localize result but it requires more careful tuning as mentioned above and better measurement model. If at any point of your project you find making panorama to be unnecessary, feel free to jump to other ideas. An interesting idea brought up by Dr. Neller is that we can make our own stitcher and only stich images partially when needed. ie. say we localized to be at a certain image, we can then stitch this image with its neighbors and get a more accurate result. Good luck!