The idea of the unity of audition and vision has been a topic of interest since the time of ancient Greece. Investigation of this idea has continued to grow in many areas, especially in psychology, music, visual art, and computer science, most notably since the invention of the world's earliest color organ. Recent developments of digital computing systems have made the concept more tangible. Currently, many audio-to-visual performance software applications are available, but they tend to depend on an arbitrary or personal association rather than perceptually significant information in transforming auditory properties into visual characteristics. This paper presents an empirical investigation of the relationship between four auditory parameters - pitch, amplitude, timbre, and duration - and four visual parameters - color, location, shape, and size - in the multimedia context. The present study also presents five audio-to-visual instruments that allow sound-to-image transformation based on the experimental results. Implementing the audio-to-visual instruments also involves real-time sound analysis by using a constant-Q transform and image generation in a Max/MSP/Jitter environment.

Last modified

• 09/16/2018

Creator

• Mikyung Kim

DOI

• https://doi.org/10.21985/N2CF1S

Subject

• Music Theory

Keyword

• match between auditory and visual parameters
• audio-to-visual instruments
• transformation algorithms
• constant-Q filter bank analysis

Date created

• 2008-10-22

Resource type

• Dissertation

Rights statement

• In Copyright