Augmented Reality using a Neural Network

This is the project that I have worked on for senior seminar at Ripon College. It is a simple augmented reality system using a neural network inspired by the biological vision system. It uses

  • Webcam as a visual sensor
  • OpenCV for feature detection and extraction,
  • A self-organizing map (SOM) for picture recognition, and
  • Microsoft WPF 3D for 3D rendering

For image recognition, the global representation of images are used as image features. The SOM, also known as Kohonen Map, is a two-layer artificial neural network comprising the input layer and the competition layer that is a grid of neurons. Each neuron is represented by a vector that has the same dimension as an input feature vector. During the training, the SOM groups the similar pictures together in the competition layer and chooses a neuron for each group.

Screenshot

Paper
Source Code
Slides