Click the play buttons above to listen to a child with autism exhibit frustration or excitement. The spectrograms of the recordings, which were made with the LENA System, show the similarities and differences in the sounds. Courtesy of Catherine Lord, Weill Cornell Medical College, Cornell University.
A new algorithmic approach to quantifying language production in individuals with autism has the potential to fill a vexing gap in autism research: the shortage of objective measures of whether a proposed therapy truly improves behavioral outcomes. Mark Clements, an engineering professor at the Georgia Institute of Technology, is building mathematical tools that can analyze audio recordings obtained using a wearable recording device known as the LENA (Language Environment Analysis) System.
The amount and quality of language an individual produces is one of the few behavioral outcomes that all researchers agree is clinically meaningful. But high-quality voice recordings of individuals going about their daily lives are difficult to collect, and then must typically be transcribed by hand, which is time-consuming and expensive. These limitations have greatly reduced the amount of data available to researchers.
Now, Clements and other researchers supported by the Simons Foundation are exploring how to use the LENA system to overcome these obstacles. If their team can automate collection and analysis of language data, "it would mean measurement of language production would be scalable, which is really important in clinical trials," says Pamela Feliciano, senior scientist at the Simons Foundation Autism Research Initiative. "It would be a huge boon."
LENA, created by the LENA Research Foundation, based in Boulder, Colo., attaches to a child's clothing and records his or her sound environment. LENA's developers have created proprietary software that quantifies vocalizations such as squeals and grunts in young children, and that shows promise in identifying toddlers at a heightened risk of developing autism.
LENA has potential for studying older children as well, says psychologist Catherine Lord, director of the Center for Autism and the Developing Brain at New York-Presbyterian Hospital and principal investigator of the current study. But LENA's proprietary software does not perform especially well at analyzing older children's vocalizations. So Clements is developing mathematical techniques for extracting meaningful communication features from recordings of older children with autism. Most speech-processing programs, such as those available in smartphones, are optimized for adults speaking in well-controlled, quiet conditions. By contrast, LENA recordings capture "competing siblings, parents, TV, all kinds of things," Clements says. Analyzing such complex recordings is "pretty much an untapped research area."
Clements' team has developed software that can identify, with about 90 percent accuracy, when the child with autism is the one speaking, and that can identify laughs and cries. The software also looks at resonant frequencies in the vocal tract to figure out whether the child is happy, sad or angry. "We've had to invent our own mathematics to do that," Clements says.
It's essential, the researchers agree, for psychologists and engineers to work together. Lord was able to tell Clements, for instance, that a simple measure such as how often a child screams might not be that informative, as children with autism often scream from happiness as well as anger. "The psychologists tell us what they think is important, and then we try to make it happen," Clements says.
