Speech Separation by Humans and Machines

The "cocktail-party effect" - the ability to focus on one voice in a sea of noises - is a highly sophisticated skill that is usually effortless to listeners but largely impossible for machines. Investigating and unraveling this capacity spans numerous fields including psychology, physiology, engineering, and computer science. All these perspectives are brought together in this volume which, for the first time, provides a comprehensive and authoritative discussion of our understanding of how humans separate speech, and the state of the art in approaching these abilities with machines.

This material is drawn from an October 2003 workshop, sponsored by the National Science Foundation, on speech separation. Leading authorities from around the world were invited to present their perspectives and discuss the points of contact to other perspectives. The result is a clear and uniform overview of this problem, and a primer in what is emerging as an important, active and successful area for the development of new techniques and applications.

Chapters include historical and current summaries of relevant research in behavioral science, neuroscience and engineering, along with more in-depth descriptions of several of the most exciting current research projects and techniques, including the latest experimental results illuminating how listeners organize the mixtures of sound they hear, and the most powerful and successful signal processing and machine learning techniques for the separation of real-world recordings of sound mixtures by one or more microphones.

There is no comparable collection that seeks to bring together the underlying experimental science and the wide variety of technical approaches to give an integrated picture of the problem and solutions to speech separation. Those specializing in speech science, hearing science, neuroscience, or computer science and engineers working on applications such as automatic speech recognition, cochlear implants, hands-free telephones, sound recording, multimedia indexing and retrieval will find Speech Separation by Humans and Machines a useful and inspiring read.

Zusammenfassung
There is a serious problem in the recognition of sounds. It derives from the fact that they do not usually occur in isolation but in an environment in which a number of sound sources (voices, traffic, footsteps, music on the radio, and so on) are active at the same time. When these sounds arrive at the ear of the listener, the complex pressure waves coming from the separate sources add together to produce a single, more complex pressure wave that is the sum of the individual waves. The problem is how to form separate mental descriptions of the component sounds, despite the fact that the mixture wave does not directly reveal the waves that have been summed to form it. The name auditory scene analysis (ASA) refers to the process whereby the auditory systems of humans and other animals are able to solve this mixture problem. The process is believed to be quite general, not specific to speech sounds or any other type of sounds, and to exist in many species other than humans. It seems to involve assigning spectral energy to distinct auditory objects and streams that serve as the mental representations of distinct sound sources in the environment and the patterns that they make as they change over time. How this energy is assigned will affect the perceived n- ber of auditory sources, their perceived timbres, loudnesses, positions in space, and pitches.

Inhalt
Speech Segregation: Problems and Perspectives.- Auditory Scene Analysis.- Speech separation.- Recurrent Timing Nets for F0-based Speaker Separation.- Blind Source Separation Using Graphical Models.- Speech Recognizer Based Maximum Likelihood Beamforming.- Exploiting Redundancy to Construct Listening Systems.- Automatic Speech Processing by Inference in Generative Models.- Signal Separation Motivated by Human Auditory Perception: Applications to Automatic Speech Recognition.- Speech Segregation Using an Event-synchronous Auditory Image and STRAIGHT.- Underlying Principles of a High-quality Speech Manipulation System STRAIGHT and Its Application to Speech Segregation.- On Ideal Binary Mask As the Computational Goal of Auditory Scene Analysis.- The History and Future of CASA.- Techniques for Robust Speech Recognition in Noisy and Reverberant Conditions.- Source Separation, Localization, and Comprehension in Humans, Machines, and Human-machine Systems.- The Cancellation Principle in Acoustic Scene Analysis.- Informational and Energetic Masking Effects in Multitalker Speech Perception.- Masking the Feature Information In Multi-stream Speech-analogue Displays.- Interplay Between Visual and Audio Scene Analysis.- Evaluating Speech Separation Systems.- Making Sense of Everyday Speech: a Glimpsing Account.