The Future of Music Translation: Transforming Images into Sound

By Carolyn Heneghan

An Indiana University professor has made a revolutionary discovery in the realm of sound and music translation and preservation. In short, IU music historian Patrick Feaster, co-founder of the First Sounds Initiative, has found a way to translate and create sound based on the record grooves found in a magazine photo.

The Historical Recording

This feat is crucial for the exploration of the oldest sound recordings researchers can find. This particular recording dates back to 1889 and is now officially the oldest gramophone recording available today (though there are earlier recordings from Thomas Edison’s phonograph).

On the recording, Feaster heard the voice of Emile Berliner, the German-born American inventor of the gramophone as he tested his latest invention. He found the source of the recording in an audio magazine advertisement that introduced consumers to what this new technology would look like. In the photo, Feaster found the grooves of the record, which he was able to translate to produce this historical recording.

To create the recording, Feaster scanned the photo into a photo editing program to straighten the round grooves into straight lines. He zoomed in closely to find a rough version of sound waves embedded in the record’s grooves. These sounds waves became the fodder for the modern recording he produced. In the recording, Berliner recited a well-known German poem as his sound test.

What This Means for Music

Feaster’s recording is not the first discovery of its kind, but its location in recording history is of great importance. The earliest known recording of music is from a 5-inch by 15-inch strip of tinfoil scored with a series of dots by a stylus on a rotating cylinder, used on Thomas Edison’s recording device on June 22, 1878. The recording includes a short cornet solo, a man reciting nursery rhymes and bouts of laughter in between.

As the ability to produce historical sounds progresses, listeners will develop a deeper appreciation for century-old music and recording processes. One might expect a revival of this time period’s music, particularly for researchers and ethnomusicologists, in the classroom, books, magazines and the like. Exciting discoveries such as this often spur more research in that particular area, so it would not be surprising if similar findings continued to surface over the coming years.

Perhaps even more exciting is the ability to translate sounds from historical images. The implications of such an ability mean endless possibilities for the future of music translation. In terms of photos, this means new sources of music from at least to as far back as the earliest history of photography.

Modern Image-Music Translation

But this concept is not one bound by history. Many modern music programs have the capability of translating images to music, not just based on an image of grooved sound waves.

One modern application of this capability is from Robin “Scanner” Rimbaud, an experimental electronic artist. Back in 2001, he sought a new way to make his music come to life through scanned images. On his album Wave of Light by Wave of Light, he used the program Metasynth—half Photoshop, half synthesizer—to innovate the world of digital sampling. On the images, where there is a darker patch, the sound is louder. Where there is more of the color red in an image, the sound moves to either the left or right-hand side of the stereo signal, and so on. He used this program to develop a full album of songs governed by the light and color of the images he chose.

If this was done in 2001, imagine what programs are able to do more than a decade later.

For a slightly different take on this concept, the program Photosounder is an audio editor and synthesizer that both translates images into and sounds and sounds into images. The company calls its program “the ultimate bridge between the graphical world and the audio world, bringing the full power of image editing to the service of creating and transforming sounds.” Instead of a typical synthesizer and sound editing program, Photosounder takes an entirely visual approach to the process.

Another such development aims to aid the blind with seeing images through the brain images created by listening to certain sounds and music. Dr. Amir Amedi of the Hebrew University of Jerusalem created the device EyeMusic, which uses a camera to survey an image and process that image through a computer to transform it into sounds and musical notes. These musical notes can paint a picture in the user’s brain. There is training involved for people to teach their brains to recognize these notes as images, but the ability of such a machine is still a remarkable achievement in terms of both music translation and boosting the brain sight of the blind.

Clearly, the link between image and sound is one that has been not only on the minds of researchers but musicians and producers as well. Whether it is the historical recording of a century-old cornet solo or a modern electronic album, the line between image and sound continues to be blurred with further developments and discoveries. Image-music synthesis will continue to grow, change and be explored, and the future of capturing music as images and images as music will always be an exciting point on the horizon.