Princeton University Thesis. Advisor: Szymon Rusinkiewicz
With input from NASA JPL, I used photogrammetry and machine learning to build an immersive reconstruction of the Mars surface from images taken by the Curiosity rover. The experience is a prototype for astronaut training and mission operations applications.
An image colorization convolutional neural network was trained using full color rover images. Then grayscale navigation images (which are more plentiful and cover more of the Martian surface) were colorized and reconstructed into 3D models using structure-from-motion and multi-view stereo. The environments were then stitched together into an immersive VR training experience.