Advanced computational technologies will help University of Miami Frost Institute for Data Science and Computing (IDSC) researchers better understand social interactions in children with autism spectrum disorder.
“We hope our ongoing studies will lead to new strategies for parents, teachers, and therapists to help children with this complex disorder,” said Daniel Messinger, PhD, director, IDSC Social and Behavioral Data Science; professor of psychology, pediatrics, electrical & computer engineering, and music engineering; coordinator, Developmental Psychology Program; and research director, Linda Ray Intervention Center at the University of Miami.
“Children with ASD have pervasive difficulties with social communication, said Messinger. “To understand their interactions with peers and preschool teachers, we use automated sensing devices in the classroom. IDSC’s supercomputing resources are invaluable for finding patterns in the vast quantities of data generated by these sensors.”
A new grant from the Simons Foundation will support the next step in the groundbreaking ASD research being conducted by Messinger. He is the principal investigator in the collaborative two-year study, “Multimodal, Objective Assessment of the ASD Phenotype: Longitudinal Stability and Change Across Contexts,” along with researchers from University of California Los Angeles (UCLA) and University of Southern California (USC).
The study will enroll 150 three- to five-year-old children to investigate how objective measurements of ASD-related behaviors, such as movement, speaking, and conversational turn-taking, are associated with the disorder’s severity. “We will study them throughout the school year to determine the stability of ASD-relevant behaviors and whether they change with time in both a relatively controlled clinical assessment and in relatively unconstrained preschool,” Messinger said.
Children’s behavior will be recorded with synchronized video and audio recorders. State-of-the-art machine learning approaches will be used to objectively characterize each child’s vocalizations, identify social gaze and smiles, and quantify social contact and social approach (digital proxemics).
The Simons Foundation’s mission is to advance the frontiers of research in mathematics and the basic sciences. Since its founding in 1994 by Jim and Marilyn Simons, the foundation has been a champion of basic science through grant funding, support for research, and public engagement.
A New Research Tool
To better understand interactions involving children with ASD, Messenger is testing the use of Whisper from OpenAI, an automatic speech recognition (ASR) system that uses machine learning to identify context and as well as spoken words.
“We have been very impressed with the language capabilities of Whisper to recognize and transcribe speech,” he said. “We have a supplement to a grant from the National Institutes of Health (NIH) to support Juan Londoño, a post-baccalaureate researcher who has been instrumental in making this project happen.”
Messenger noted that Whisper goes well beyond older technologies that simply count vocalizations, to assess the diversity of words used by a child as well as the number of phonemes (individual sounds in a word). “This tool is opening up a new pipeline for our work, and our preliminary results have been very favorable.”
