Percept

{Percept} is based on a pen and paper exercise I created years ago that was turned into a an interactive multimedia installation during a collaboration with William Brent in early 2006.

The piece uses 4 walls of projected video and 4 channels of spatialized audio to create a immersive environment that completely surrounds the visitor. Sound content is provided by visitors to the installation, who are asked to contribute by interpreting abstract concepts with sound. Sound and video are processed, and all aspects of the installation evolve according to how visitors interact with the system.

Technical Notes

Wrangling 4 walls of realtime interactive video, adaptive audio processing and surround spatialization, not to mention user interface and voting functions, all requires a lot of computing power. To accomplish this, {Percept} uses 6 networked computers, all of which share data, report to each other on their status and continually adapt to the actions of visitors. Components for the installation are shared by machines whenever possible (the user interface, voting system, and system logic are all hosted on one machine for example), but further consolidation proved impossible within the limits of our hardware.

The innards: Max/MSP and Jitter is used for video processing, audio recording and playback, sound synthesis, user interfaces, system logic and health monitoring. Audio processing and spatialization are done in Suppercollider. Communication between machines takes place over switched ethernet via CNMAT's Open Sound Control. I put together a fairly comprehensive diagram for the entire system, which you can view to the left.

Video Clips

Taken at Collision Symposium, 2006.

Percept Move (hi res, 25MB) | Percept Movie (low res, 7MB)