Voice and artificial intelligence experts gathered at the inaugural Voice AI Symposium held in Washington DC last month to share insight and safeguards for the use of voice as a biomarker for health. The one-day interactive event brought together experts in the field of voice biomarkers, including industry, startups, academia, researchers, patients, patient advocacy groups and underserved and underrepresented communities. These experts presented research and discussed the impact voice can have on treating diseases as varied as lung cancer, Parkinson’s disease, heart disease, and mental health.

The Voice AI Symposium was a unique, first-of-its-kind gathering of experts across industries to explore the impact voice can have on health care and was hosted by Bridge2AI-Voice Consortium, a program based on the NIH-funded study called Voice as a Biomarker of Health that is co-led by Dr Yaël Bensoussan, MD, from USF Health Morsani College of Medicine and Olivier Elemento, PhD from Weill Cornell Medicine, who are co-principal investigators for the project.

Dr. Yaël Bensoussan and Dr. Olivier Elemento.

Learn more about the Voice AI Symposium — more…

Photos by Peter Cleary, Weill Cornell College of Medicine.

 As one of four data generation projects funded by NIH’s Bridge2AI, this multi-institution project – co-led by the University of South Florida and Weill Cornell Medicine – will bring together medical, voice, AI, engineering, and ethics experts to create a human voice database using privacy-preserving AI, giving doctors a new tool for diagnosing conditions known to have associations with voice alterations.

Tampa FL (Sept 13, 2022) – Artificial intelligence may soon help doctors diagnose and treat diseases, including cancer and depression, based on the sound of a patient’s voice, as 12 leading research institutions launch a landmark National Institutes of Health-funded academic project that may establish voice as a biomarker used in clinical care.

The University of South Florida in Tampa, FL, is the lead institution on the project in collaboration with Weill Cornell Medicine in New York City, 10 other institutions in the United States and Canada, and French-American AI biotech startup Owkin. The first year of the project includes $3.8 million from the NIH, with subsequent funding over the following three years contingent upon annual NIH appropriations by Congress that could bring the overall award to $14 million.

Called Voice as a Biomarker of Health, the project is one of several recently funded by the NIH Common Fund’s Bridge2AI program, which is designed to use AI to tackle complex biomedical challenges. The voice project will aim to build an ethically sourced database of diverse human voices while protecting patient privacy. Using this data, machine learning models will be trained to spot diseases by detecting changes in the human voice, which could empower doctors with a low-cost diagnostic tool to be used alongside other clinical methods.

Based on the existing literature and ongoing research, the research team has identified five disease cohort categories for which voice changes have been associated with specific diseases with well-recognized unmet needs. Data collected for this project will center on the following disease categories:

• Voice disorders: (laryngeal cancers, vocal fold paralysis, benign laryngeal lesions)
• Neurological and neurodegenerative disorders (Alzheimer’s, Parkinson’s, stroke, ALS)
• Mood and psychiatric disorders (depression, schizophrenia, bipolar disorders)
• Respiratory disorders (pneumonia, COPD)
• Pediatric voice and speech disorders (speech and language delays, autism)

Although preliminary work with voice data has been promising, limitations to integrating voice as a biomarker in clinical practice have been linked to small datasets, ethical concerns around data ownership and privacy, bias and lack of diversity of the data. To solve these, the Voice as a Biomarker of Health project is creating a large, high-quality, multi-institutional and diverse voice database that is linked to identity-protected/unidentifiable biomarkers from other data, such as demographics, medical imaging and genomics. Federated learning technology – a novel AI framework that allows machine learning models to be trained on data without the data ever leaving its source – will be deployed across multiple research centers by Owkin to demonstrate that cross-center AI research can be conducted while preserving the privacy and security of sensitive voice data.

Supported by AI experts, bioethicists and social scientists, the project aims to transform our fundamental understanding of diseases and introduce a revolutionary new method of diagnosing and treating diseases into clinical settings. As the human voice is low-cost, easy to store and readily available, diagnosing diseases through the voice using AI could prove a transformative step in precision medicine and accessibility.

Yael Bensoussan, MD, from USF Health Morsani College of Medicine, is a co-principal investigator for the NIH’s Bridge2AI, a multi-institution project. 

Voice as a Biomarker of Health is being co-led by Dr Yaël Bensoussan, MD, from USF Health Morsani College of Medicine and Olivier Elemento, PhD from Weill Cornell Medicine, who are co-principal investigators for the project. The project also includes lead investigators from 10 other universities in North America; Alexandros Sigaras and Anaïs Rameau (Weill Cornell Medicine), Maria Powell (Vanderbilt University Medical Center), Ruth Bahr and Gaetane Michaud (USF Health Morsani College of Medicine),  Alistair Johnson (Hospital for Sick Children), Philip Payne (Washington University in St. Louis), David Dorr (Oregon Health & Science University), Jean-Christophe Belisle-Pipon (Simon Fraser University), Vardit Ravitsky (University of Montreal), Satrajit Ghosh (Massachusetts Institute of Technology), Kathy Jenkins (Boston Children’s Hospital), Frank Rudzizc (University of Toronto) and Jordan Lerner-Ellis (Sinai Health). AI startup Owkin is deploying its federated learning technology across multiple research institutions to protect the security and privacy of sensitive voice data.

“Voice has the potential to be a biomarker for several health conditions,” said Dr. Bensoussan, assistant professor in the Department of Otolaryngology and director of USF Health Voice Center at the USF Health Morsani College of Medicine. “Creating an effective framework that incorporates huge datasets using the best of today’s technology in a collaborative manner will revolutionize the way that voice is used as a tool for helping clinicians diagnose diseases and disorders.”

“The potential for using voice and sounds together with advanced AI algorithms to accurately diagnose certain diseases is incredible. Our future findings could lead to a revolution in health care where continuous voice monitoring could alert physicians earlier than currently possible to certain conditions, such as infections or neurological diseases,” said Dr. Elemento, director of the Englander Institute for Precision Medicine and a professor of physiology and biophysics at Weill Cornell Medicine.

Olivier Elemento, PhD, from Weill Cornell Medicine, is a co-principal investigator for the NIH’s Bridge2AI, a multi-institution project.

“By using AI to analyze minute changes in the human voice, we aim to help doctors to diagnose and treat diseases ranging from cancer to depression. Vocal biomarkers are set to play an increasingly important role in healthcare. We are excited to be using federated learning, our privacy-preserving AI framework, to connect the medical world together in the pursuit of improving outcomes for patients,” said Thomas Clozel MD, co-founder and CEO of Owkin.

This project is supported by the National Institutes of Health award number: OT2OD032720.

About USF Health

USF Health’s mission is to envision and implement the future of health. It is the partnership of the USF Health Morsani College of Medicine, the College of Nursing, the College of Public Health, the Taneja College of Pharmacy, the School of Physical Therapy and Rehabilitation Sciences, the Biomedical Sciences Graduate and Postdoctoral Programs, and USF Health’s multispecialty physicians group. The University of South Florida is a high-impact global research university dedicated to student success. Over the past 10 years, no other public university in the country has risen faster in U.S. News and World Report’s national university rankings than USF. For more information, visit health.usf.edu

About Weill Cornell Medicine
Weill Cornell Medicine is committed to excellence in patient care, scientific discovery and the education of future physicians in New York City and around the world. The doctors and scientists of Weill Cornell Medicine — faculty from Weill Cornell Medical College, Weill Cornell Graduate School of Medical Sciences, and Weill Cornell Physician Organization — are engaged in world-class clinical care and cutting-edge research that connect patients to the latest treatment innovations and prevention strategies. Located in the heart of the Upper East Side’s scientific corridor, Weill Cornell Medicine’s powerful network of collaborators extends to its parent university Cornell University; to Qatar, where Weill Cornell Medicine-Qatar offers a Cornell University medical degree; and to programs in Tanzania, Haiti, Brazil, Austria and Turkey. Weill Cornell Medicine faculty provide exemplary patient care at NewYork-Presbyterian/Weill Cornell Medical Center, NewYork-Presbyterian Westchester Behavioral Health Center, NewYork-Presbyterian Lower Manhattan Hospital, NewYork-Presbyterian Queens and NewYork-Presbyterian Brooklyn Methodist Hospital. Weill Cornell Medicine is also affiliated with Houston Methodist. For more information, visit weill.cornell.edu.

Media contacts:

• University of South Florida: Allison Long: allison65@usf.edu
• Weill Cornell Medicine: Eliza Powell: elp4014@med.cornell.edu
• Owkin: Alexander Blackburn: blackburn@owkin.com
• NIH: Rachel Britt: cfcomms@nih.gov

Photos: Yael Bensoussan, MD (Allison Long/USF Health) Olivier Elemento, PhD (WCM /Travis Curry)

Our voices are one of the most important ways we communicate. Be it projecting to the audience while giving a presentation, encouraging your children on the field or hitting the right notes when you sing, a healthy voice is vital to everyday life. And that means voice disorders can be far more than just a health care nuisance.

Fortunately, the USF Health Voice Center is here to help. The center treats patients with disorders of the upper airway, voice and swallowing.

Yael Bensoussan, MD, MSc

“Voice disorders can really impact patients’ lives as our voices are so important for communicating, socializing and working, especially in the new “zoom era,”‘ said the center’s director, Yael Bensoussan, MD, MSc, who is also an assistant professor of Otolaryngology in the Morsani College of Medicine.

Common symptoms for voice or swallowing disorders include:

• A quivering sound when you talk or sing
• Hoarseness ​for more than three weeks
• A strained or choppy voice
• A weak or whispery voice
• A voice that involuntarily changes in pitch or may be too high or low
• Pain or difficulty when swallowing
• ​Food getting stuck when swallowing
• Choking or coughing while eating

The USF Health Voice Center is Tampa Bay area’s only academic multidisciplinary voice center. A typical visit may involve an examination by both a laryngologist and a medical speech pathologist.

“I’m very lucky to have a speech pathologist with me,” Dr. Bensoussan said. “Patients always see both of us, so they can get both of our expertise.”

Stephanie Watts, PhD, SLP

Stephanie Watts, PhD, SLP, is the clinical speech pathologist that works closely with Dr. Bensoussan.

“The model of our practice is to combine surgical, medical and behavioral intervention in a truly collaborative effort,” Dr. Watts said. “We really enjoy seeing patients. We create a journey with our patients.”