Breaking The Sound Barrier

A Drexel lab blurs the line between music making and listening.

PHILADELPHIA – In a windowless lab at Drexel University’s College of Engineering, Assistant Prof. Youngmoo Kim and his students think up ways to put the power of a music arranger into the hands of unskilled and untutored listeners.

Literally into their hands: Soon, if Kim’s research pans out, an iPhone could be all a listener needs to imprint downloaded music with his or her own taste and style, adjusting tempo, pitch, and mood. “Think of it as the 21st-century version of the mixed tape,” he says.

Kim, a baritone who sang with the Tanglewood Festival Chorus while earning a Ph.D. at MIT, works at the busy intersection of music and electrical and computer engineering, where a laptop is the new sound studio and listening innovations whizz by in a blur.

Already, open-source and commercial software enables young DJs to post online mash-ups and re-edit everything from the Beatles to Beyoncé. Kim wants to expand the boundaries of accessibility further, making the listener an interactive participant in music making.

He starts by turning music into math. With software developed using signal processing algorithms, computers can “hear” a recording, getting a mathematical representation of a sound’s pitch and duration, as well as of distinct units of speech. Kim is working to fine-tune the computer’s listening ability so it will grasp more of the complexity of sound. From there, he and undergraduates and graduates at Drexel’s Music Entertainment Technology lab (METlab) will develop software to enhance the human listener’s experience.

One project speeds the search of downloaded recordings — which in the case of 20-somethings can fill a dozen or more gigabytes on an MP3 player. A listener who enjoys a particular tune could prolong the enjoyment by quickly locating one with similar instrumentation and timbre.

Another research effort would give a handheld device the capability of a sound technician, letting a listener mix and create a unique musical production. The result, Kim says with infectious enthusiasm, would lower barriers to musical expression. Already, professional producers can remix a recorded song in interesting ways. “What we could provide is a way for a nontechnical person to explore the possibilities.” The music industry might respond in kind, he suggests, releasing not just completed arrangements of a song but separately recorded tracks of each instrument and voice, or recordings of varied concert performances. A single song could then become an expanding palette, allowing listeners to mix musical colors at whim.

METlab is also working to identify moods in music, in part through one of Kim’s favorite data-gathering and teaching tools: a game. In this case, it’s the METlab-developed, collaborative MoodSwings, designed to capture, then quantify players’ consensus on whether music is happy or sad, intense or laidback.

Kim notes that more work is needed to correct major gaps in the software. Take, for instance, the “cocktail party” challenge: A computer can’t isolate individual voices in a crowded room; nor can it distinguish between foreground and background sounds from an orchestra. Discerning a quality performance is another problem. “What we perceive as good musicianship is a long way from what we can quantitatively measure,” says Kim.

Kim believes that METlab’s listener tools won’t devalue musical virtuosity. It’s a subject he understands, having begun his own musical training with the violin at age 5 and gone on to earn two music degrees.

Kim, whose lab is active in K-12 outreach, also knows that the training he got is out of reach of many youngsters in an era when school districts are reducing music and art programs. Thus, he sees strong educational potential in the new software, providing kids a route to musical composition. Even popular video games such as Guitar Hero and Rock Band can help develop a young person’s musical skill, Kim hypothesizes. Patrick Richardson, a Drexel engineering graduate student who is also a drummer, is designing a study to test the theory, which, if proven, offers “an incredible tool for music education,” Kim says. And who knows? Inspired teens might take up an instrument or learn to write their own tunes.

To play a METlab game, go to http://schubert.ece.drexel.edu/.

Mark Matthews is managing editor of Prism.