Scientists sound warning about use of autism speech detector

Post Reply New Topic

A popular system used to detect speech does not identify most vocalizations made by autistic children aged 5 and older, a new study suggests1. The findings put a damper on plans to use the system for tracking speech development in these children.

Researchers typically assess children’s speech through labor-intensive, manual transcriptions of audio recordings. By contrast, the system, called LENA (Language Environment Analysis), consists of a wearable audio recorder paired with software that labels each recorded sound with its source.

LENA’s creators designed the system to pick out the voices of children aged 4 and under among those of adults and ambient sounds. It tracks how frequently a child speaks and how often she participates in back-and-forth exchanges with an adult.

A 2010 study found that LENA can distinguish autistic children aged 16 to 48 months from typical children based on their speech patterns2. The system garnered intense interest from researchers, including for assessing language therapies.

The findings suggest that LENA’s use in these trials should be limited because it cannot distinguish older children’s voices from those of adults.

“The pitch difference is very wide between a child’s voice versus an adult’s voice, but as you get older, the pitch overlaps,” Tager-Flusberg says. “Then it becomes much harder for LENA to discriminate.”

No alternatives to LENA for older children exist, but these sobering results may spark some research on that front, others say.

Tager-Flusberg’s team has since 2013 used LENA in older children to record their speech. The team noticed that the software misses many of the children’s utterances. Rebecca Jones and her colleagues made similar observations.

The two teams joined forces to track the problems; they analyzed LENA recordings from two sets of autistic children aged 5 to 18 years who wore clothes designed to hold the recorder in a chest pocket.

Tager-Flusberg’s team recorded 15 minimally verbal children with autism during diagnostic evaluations. For every child, they transcribed three five-minute intervals, each beginning with a child’s vocalization. They compared their assessments with the labels LENA assigned.

Jones and her colleagues analyzed data from 36 autistic children with a range of verbal abilities. The children wore the device at home three days a week for one or eight weeks, and also as they completed assessments at a clinic. The team manually transcribed everything the software flagged as speech and analyzed how often their transcriptions agreed with LENA’s labels.

Both teams found that the software detects less than half of a child’s utterances, mostly because it mistakes children’s voices for adult ones. And the older the child, the less accurate the software.

The use of two different methods of data collection and analysis “really adds to the study’s rigor,” says David Trembath, associate professor of speech pathology at Griffith University in Brisbane, Australia, who was not involved in the study.

“These automated approaches to data analysis have huge potential, but we need to do the work to make sure that they’re reliable,” he says.

The results do not surprise anyone familiar with LENA.

“I would have been surprised if it did work,” says Steven Warren, distinguished professor of speech-language-hearing at the University of Kansas in Lawrence and scientific adviser for the LENA Foundation, which makes the tool. LENA isn’t set up to work in children older than 4 years, Warren says.

Jones and her colleagues are using their transcriptions to train a machine-learning algorithm to distinguish between vocalizations of older children and adults. Their preliminary results suggest that the algorithm can accurately distinguish the voices more than 80 percent of the time.