As one of four data generation projects funded by NIH’s Bridge2AI, this multi-institution project – co-led by the University of South Florida and Weill Cornell Medicine – will bring together medical, voice, AI, engineering, and ethics experts to create a human voice database using privacy-preserving AI, giving doctors a new tool for diagnosing conditions known to have associations with voice alterations.

Tampa FL (Sept 13, 2022) – Artificial intelligence may soon help doctors diagnose and treat diseases, including cancer and depression, based on the sound of a patient’s voice, as 12 leading research institutions launch a landmark National Institutes of Health-funded academic project that may establish voice as a biomarker used in clinical care.

The University of South Florida in Tampa, FL, is the lead institution on the project in collaboration with Weill Cornell Medicine in New York City, 10 other institutions in the United States and Canada, and French-American AI biotech startup Owkin. The first year of the project includes $3.8 million from the NIH, with subsequent funding over the following three years contingent upon annual NIH appropriations by Congress that could bring the overall award to $14 million.

Called Voice as a Biomarker of Health, the project is one of several recently funded by the NIH Common Fund’s Bridge2AI program, which is designed to use AI to tackle complex biomedical challenges. The voice project will aim to build an ethically sourced database of diverse human voices while protecting patient privacy. Using this data, machine learning models will be trained to spot diseases by detecting changes in the human voice, which could empower doctors with a low-cost diagnostic tool to be used alongside other clinical methods.

Based on the existing literature and ongoing research, the research team has identified five disease cohort categories for which voice changes have been associated with specific diseases with well-recognized unmet needs. Data collected for this project will center on the following disease categories:

• Voice disorders: (laryngeal cancers, vocal fold paralysis, benign laryngeal lesions)
• Neurological and neurodegenerative disorders (Alzheimer’s, Parkinson’s, stroke, ALS)
• Mood and psychiatric disorders (depression, schizophrenia, bipolar disorders)
• Respiratory disorders (pneumonia, COPD)
• Pediatric voice and speech disorders (speech and language delays, autism)

Although preliminary work with voice data has been promising, limitations to integrating voice as a biomarker in clinical practice have been linked to small datasets, ethical concerns around data ownership and privacy, bias and lack of diversity of the data. To solve these, the Voice as a Biomarker of Health project is creating a large, high-quality, multi-institutional and diverse voice database that is linked to identity-protected/unidentifiable biomarkers from other data, such as demographics, medical imaging and genomics. Federated learning technology – a novel AI framework that allows machine learning models to be trained on data without the data ever leaving its source – will be deployed across multiple research centers by Owkin to demonstrate that cross-center AI research can be conducted while preserving the privacy and security of sensitive voice data.

Supported by AI experts, bioethicists and social scientists, the project aims to transform our fundamental understanding of diseases and introduce a revolutionary new method of diagnosing and treating diseases into clinical settings. As the human voice is low-cost, easy to store and readily available, diagnosing diseases through the voice using AI could prove a transformative step in precision medicine and accessibility.

Yael Bensoussan, MD, from USF Health Morsani College of Medicine, is a co-principal investigator for the NIH’s Bridge2AI, a multi-institution project. 

Voice as a Biomarker of Health is being co-led by Dr Yaël Bensoussan, MD, from USF Health Morsani College of Medicine and Olivier Elemento, PhD from Weill Cornell Medicine, who are co-principal investigators for the project. The project also includes lead investigators from 10 other universities in North America; Alexandros Sigaras and Anaïs Rameau (Weill Cornell Medicine), Maria Powell (Vanderbilt University Medical Center), Ruth Bahr and Gaetane Michaud (USF Health Morsani College of Medicine),  Alistair Johnson (Hospital for Sick Children), Philip Payne (Washington University in St. Louis), David Dorr (Oregon Health & Science University), Jean-Christophe Belisle-Pipon (Simon Fraser University), Vardit Ravitsky (University of Montreal), Satrajit Ghosh (Massachusetts Institute of Technology), Kathy Jenkins (Boston Children’s Hospital), Frank Rudzizc (University of Toronto) and Jordan Lerner-Ellis (Sinai Health). AI startup Owkin is deploying its federated learning technology across multiple research institutions to protect the security and privacy of sensitive voice data.

“Voice has the potential to be a biomarker for several health conditions,” said Dr. Bensoussan, assistant professor in the Department of Otolaryngology and director of USF Health Voice Center at the USF Health Morsani College of Medicine. “Creating an effective framework that incorporates huge datasets using the best of today’s technology in a collaborative manner will revolutionize the way that voice is used as a tool for helping clinicians diagnose diseases and disorders.”

“The potential for using voice and sounds together with advanced AI algorithms to accurately diagnose certain diseases is incredible. Our future findings could lead to a revolution in health care where continuous voice monitoring could alert physicians earlier than currently possible to certain conditions, such as infections or neurological diseases,” said Dr. Elemento, director of the Englander Institute for Precision Medicine and a professor of physiology and biophysics at Weill Cornell Medicine.

Olivier Elemento, PhD, from Weill Cornell Medicine, is a co-principal investigator for the NIH’s Bridge2AI, a multi-institution project.

“By using AI to analyze minute changes in the human voice, we aim to help doctors to diagnose and treat diseases ranging from cancer to depression. Vocal biomarkers are set to play an increasingly important role in healthcare. We are excited to be using federated learning, our privacy-preserving AI framework, to connect the medical world together in the pursuit of improving outcomes for patients,” said Thomas Clozel MD, co-founder and CEO of Owkin.

This project is supported by the National Institutes of Health award number: OT2OD032720.

About USF Health

USF Health’s mission is to envision and implement the future of health. It is the partnership of the USF Health Morsani College of Medicine, the College of Nursing, the College of Public Health, the Taneja College of Pharmacy, the School of Physical Therapy and Rehabilitation Sciences, the Biomedical Sciences Graduate and Postdoctoral Programs, and USF Health’s multispecialty physicians group. The University of South Florida is a high-impact global research university dedicated to student success. Over the past 10 years, no other public university in the country has risen faster in U.S. News and World Report’s national university rankings than USF. For more information, visit health.usf.edu

About Weill Cornell Medicine
Weill Cornell Medicine is committed to excellence in patient care, scientific discovery and the education of future physicians in New York City and around the world. The doctors and scientists of Weill Cornell Medicine — faculty from Weill Cornell Medical College, Weill Cornell Graduate School of Medical Sciences, and Weill Cornell Physician Organization — are engaged in world-class clinical care and cutting-edge research that connect patients to the latest treatment innovations and prevention strategies. Located in the heart of the Upper East Side’s scientific corridor, Weill Cornell Medicine’s powerful network of collaborators extends to its parent university Cornell University; to Qatar, where Weill Cornell Medicine-Qatar offers a Cornell University medical degree; and to programs in Tanzania, Haiti, Brazil, Austria and Turkey. Weill Cornell Medicine faculty provide exemplary patient care at NewYork-Presbyterian/Weill Cornell Medical Center, NewYork-Presbyterian Westchester Behavioral Health Center, NewYork-Presbyterian Lower Manhattan Hospital, NewYork-Presbyterian Queens and NewYork-Presbyterian Brooklyn Methodist Hospital. Weill Cornell Medicine is also affiliated with Houston Methodist. For more information, visit weill.cornell.edu.

Media contacts:

• University of South Florida: Allison Long: allison65@usf.edu
• Weill Cornell Medicine: Eliza Powell: elp4014@med.cornell.edu
• Owkin: Alexander Blackburn: blackburn@owkin.com
• NIH: Rachel Britt: cfcomms@nih.gov

Photos: Yael Bensoussan, MD (Allison Long/USF Health) Olivier Elemento, PhD (WCM /Travis Curry)