Computer Audio


Audio is a fundamental feature of PCs and hand-held devices.  In computer audio, algorithms process audio data for a variety of applications.

 

From 2004 to 2009, I taught a unique course on Computer Audio at the University of Mississippi.  Students gained an understanding of sound, speech, and music and their digital representation and processing in computer software. Programming assignments provided hands-on experience in playing, recording, mixing, synthesizing, and analyzing audio. Class lectures presented algorithms and techniques for compression, watermarking, synthesis, sonification, pitch and beat detection, audio fingerprinting, speech recognition, speaker recognition, and audio retrieval.  One of the highlights of the class was a concert performed by the students in which they demonstrated their sound synthesis programs.

 

I have had a lifelong interest in sound and music.  My first audio programming occurred in the 1980s as a hobby.  After completing my Ph.D. in 1996, I joined Graham-Patten Systems in Grass Valley, California, which was my first opportunity to work on computer audio as a professional software engineer.  At that time, Graham-Patten was interested in developing an audio database system to accompany their audio mixer.  I became interested in the problem of how to search an audio database.

 

It was clear that if text descriptions of sounds were available, those text descriptions could be searched to locate sounds.  But what if there were no text descriptions?  This occurred either because no one has the time to enter them into the computer, or because many sounds are so difficult to describe in words.  What is needed is the ability to search sounds based on their audio characteristics, independent of text descriptions and other "metadata".

 

In 1997 I invented a "sound-matching" algorithm.  Any audio recording can be analyzed and represented by one or more "signatures".  The signatures derived from two recordings can be compared to estimate the similarity of their sounds as perceived by human listeners.  The more similar are two sounds, the higher is their "similarity score".  With this algorithm, a user can present any sound as an example or prototype, and even mimic a sound into a microphone, and then automatically locate similar sounds within a collection of audio recordings.  This "sounds-like" search is a form of "content-based" audio retrieval: sounds are retrieved based on their audio content and not on text descriptions of the sounds.  This approach gives users a new way to access and explore audio collections.

 

In 1997 I became annoyed with viewing audio waveform displays, which are graphs of the amplitude (or loudness) of audio signals, because these displays provided no information about the frequency content of the audio signals, i.e., whether a sound is low (bass), mid-range, or high pitched.  And so I invented and patented the technique of coloring the waveform display to convey the frequency content of the audio signal.  The new "colored" waveform display is a significant improvement over the traditional display.  Below is an example showing the uncolored and colored waveform representations of an audio recording.  In the colored display, the sound of a cat appears in orange, followed by the sounds of a fly (lavender) and a rooster (red).  More examples can be seen here.    The colored waveform display is especially valuable in audio-editing systems, where the waveform display is the focal point of user interaction.

In June 1998, my projects at Graham-Patten Systems were spun off as a start-up company named Comparisonics Corporation and I became an entrepreneur.

 

At Comparisonics, my colleague Steve Bailey and I developed the FindSounds.com search engine for finding sound effects on the Web.  It incorporates both text search and the sounds-like search for locating audio files on the Web.  Hits are represented by colored waveform displays.  FindSounds.com launched on August 1, 2000.  It recently celebrated its 14th anniversary as the leading Web search engine for sound effects.  It currently processes each month more than 2,000,000 sound searches for more than 300,000 users.  You can try it here: enter any word or phrase (for example, "elephant" or "bass drum") and then click on the Search button.

FindSounds
Search the Web for Sounds

We received requests for a "home" version of FindSounds.com that would permit users to search their own collections of audio files using the Comparisonics technologies.  And so in 2002 we released FindSounds Palette, which is an audio player, recorder, editor, database, search engine, and Web browser, all in one software program.  Since its release, the free trial has been downloaded more than 350,000 times.


In 2011, I created FindSounds for Android, the first mobile app for audio Web search.  With this app, sound effects can be located on the Web and downloaded to an Android device, where they can be played and saved as ringtones, alarms, and other notification sounds.

 

The ability to compare sounds and measure their similarity has many applications.  For example, it can be used to monitor the sounds of machinery.  I have developed a unique general-purpose monitoring system that can listen to any environment and detect abnormal sounds in real time.  This system has widespread application in industrial environments.

 

Selected Writings


S. V. Rice and S. M. Bailey, "A System for Searching Sound Palettes," in Proceedings of the Eleventh Biennial Symposium on Arts and Technology, New London, CT, 2008 (pdf)

 

S. V. Rice, "A Survey Course on Computer Audio," Journal of Computing Sciences in Colleges, 20(6), 2005 (pdf)

 

S. V. Rice, "Frequency-Based Coloring of the Waveform Display to Facilitate Audio Editing and Retrieval," in Proceedings of the 119th Convention of the Audio Engineering Society, Paper #6530, New York, 2005 (pdf)

 

S. V. Rice and S. M. Bailey, "A Web Search Engine for Sound Effects," in Proceedings of the 119th Convention of the Audio Engineering Society, Paper #6622, New York, 2005 (pdf)

 

S. V. Rice and S. M. Bailey, "General-Purpose Real-Time Monitoring of Machine Sounds," in Essential Technologies for Successful Prognostics: Proceedings of the 59th Meeting of the Society for Machinery Failure Prevention Technology, Virginia Beach, VA, 2005 (pdf)