I want to use what I’ve learned in this class to focus on approaching the use of video and sound in unique ways. So for my final project I want to build some sort of visual and/or auditory headset that transforms the user’s surroundings in unexpected ways, perhaps enabling them to see an aspect of reality that the naked human body isn’t capable of perceiving.

In the spirit of current VR hype, I will build a headset that simulates an audio-visual synesthetic experience, for any curious individuals interested in having a temporarily destroyed sensorium while running the risk of bumping into walls, stubbing their toes or meandering out into traffic in a manner similar to how someone in the throes of an intense psychedelic experience might do so.

The headset will consist of a visual input (camera) and an audio input (sound detector) hooked up to an Arduino Uno, which will run a program that will take the sound data and output it to a color LCD display or two, and take the camera image data and output it to a standard (3.5mm) headphone jack with which the user listen to with their own headphones.

Video to audio: I will translate the color information from the camera image at each frame to tones — a tone for each color. It could either calculate the most prevalent color and translate that into one tone, or calculate all existing colors in each image and translate them into a polyphonic set of tones, or anything in between.

Audio to video: Amplitude of sound detected will translate to image opacity on the display, and pitch/frequency will translate to hue.

Bill of Materials:

System Diagram:

Roadmap:

Nov 16:

• Have purchased all parts (or at least enough to begin assembly)

Nov 23:

• Have begun writing and testing code portion
• Continue circuit assembly
• Begin circuit housing / headset assembly

Nov 30:

• Have completed code portion
• Finalizing circuit assembly, begin user testing
• Continue circuit housing / headset assembly

Dec 7:

• Apply changes from user testing
• Finalize circuit housing / headset assembly
• Prepare presentation, finalize documentation

Dec 14:

• Present