Sensing music in the New York streetscape
Parsons, The New School for Design
SoundAffects was a public installation that translated the sensory overload of a New York street into a serene audiovisual experience. Tellart created a wall with embedded cameras, sensors and lights that observed its surroundings, harmonising yellow cabs with jackhammers and evening rain. This onsite and online installation expressed Parsons' mission to teach holistic design by reimagining physical perceptions and considering complex systems.
Sensors detected multiple layers of information from the street including sound, movement, color and temperature. Multiple cameras recorded live video, both to mine for color data and for the online experience.
Tellart's in-house electronic music specialists wrote the audio composition algorithm that generated harmonious music from a cacophony of unpredictable camera and sensor inputs.
The installation included a graphical visualizer that translated massive amounts of data into abstract colour blocks, reflecting the street's activity in a simple and direct way.
SoundAffects generated significant web traffic. Online visitors viewed a live video feed with its accompanying musical and graphical composition. They could also scrub backwards through the timeline to compare the mood at different moments–from rain showers to sunny mornings. Markers in the timeline flagged noteworthy events like parades or people passing by with colorful balloons.
Our custom-designed system translated complex activity patterns into an audiovisual expression of a precise moment in space and time.
“Parsons The New School for Design, Mono and Tellart have created an installation and app that turns the sights and sounds of the city into a literal form of street music.”
“We want people to engage with their cities in a way they never have before. Taking a typical city block and shifting how we view it opens a world of possibilities for interpretation. It's a sophisticated, yet simple, way to understand the design-focused learning happening at Parsons.”
Travis Olson, Mono
“This exploration is an invitation for the public to participate in the design of our cities, and will hopefully shed some light on the complexities of urban life. It's a tangible way to understand design's role in the world around us by looking at everyday occurrences in a different way.”
Joel Towers, Executive Dean of Parsons The New School for Design
Project Data
Highlights:
• 10-day Live Streaming generative audio experiment
• A pioneering example of HTML5 Streaming available through mobile browser
• Sensor inputs included temperature readings, color analysis of video feeds, precipitation measurements, noise levels, cellular phone activity/interference, and proximity sensors.
Awarded:
Selected Press:
Project team:
Tellart designed the experience and developed the technology, working closely with creative agency Mono who developed the core concept.
Sign up for our newletter