Beyond Area Under the Receiver Operating Characteristic Curve: Evaluating Predictive Performance Metrics Under Class Imbalance in Real-World Clinical Data
Background: Predictive models increasingly support clinical decision-making, although imbalanced outcome distributions are common in health care datasets and can distort performance evaluation. The area under the receiver operating characteristic curve (AUROC) remains the most frequently reported metric, despite its limited ability to reflect clinically meaningful performance under class imbalance. Objective: This study aimed to examine the influences of metric selection on the clinical interpretation of predictive models in imbalanced real-world health care data. Methods: This was a retrospective cohort study, including 17,018 hospitalized patients with COVID-19. Two predictive models using extreme gradient boosting (XGBoost) were developed to predict kidney replacement therapy (KRT) and mortality. Model performance was assessed using AUROC, macro–score, class-specific precision and recall, calibration (curve, slope, and intercept), decision curve analysis, and learning curves. Standard rebalancing strategies were applied exclusively to the training data to evaluate their impact on performance. Results: KRT occurred in 9.5%, and mortality in 18.0%. Although AUROC values were high (0.928 for KRT and 0.945 for mortality), performance in the minority class was substantially lower. For KRT, precision was 0.539 and recall 0.372; for mortality, precision was 0.725 and recall 0.718. Rebalancing strategies were associated with higher recall for the minority class, but this gain was accompanied by a reduction in precision, with minimal impact on AUROC values. As a result, AUROC remained high despite clinically relevant changes in error distribution between false positives and false negatives. The learning curves show a plateau-like shape, with stable validation performance across all training set sizes for both outcomes. Conclusions: AUROC alone is insufficient to evaluate prediction models in imbalanced health care scenarios, even with rebalancing. Routine reporting of class-aware metrics, alongside learning curve analysis, is essential to support robust and clinically meaningful evaluation of predictive models, rather than their direct translation into practice.
<img src="https://jmir-production.s3.us-east-2.amazonaws.com/thumbs/e0a2583c49e29d41efcc3b688510ec53" />
First-in-Human Stem Cell Therapy Trial for Huntington’s Disease Begins at UCI Health
The world’s first in-human embryonic stem cell-derived clinical trial for Huntington’s disease has launched at UCI Health, the clinical arm of the University of California, Irvine. The Phase Ib/IIa trial will evaluate the safety of hNSC-01 neural stem cells derived from embryonic stem cells delivered to the brain by a specialized neurological mapping and targeting stereotactic system.
Huntington’s disease is a fatal, progressive genetic disorder that gradually destroys brain cells. It usually begins between the ages of 35 and 50 with symptoms that include involuntary movements, difficulty thinking and planning daily tasks, and mood changes such as depression. If successful, this therapy could prolong independent living and significantly reduce long-term care costs.
“This clinical trial highlights the important role that an interdisciplinary academic and clinical team together with the HD families, plays in advancing medicine,” said Leslie M. Thompson, PhD, professor of psychiatry and human behavior at UC Irvine. “We are grateful to our patients and their incredible families for their bravery to provide hope for others with very few options.”
The first patient received the intervention at UCI Health Irvine (home to Orange County’s first adult bone marrow/stem cell transplant and cellular therapy program) in May. A second patient is scheduled to receive the intervention in July.
“The first patient intervention went very well. To date, they haven’t reported any serious adverse events,” said Ravi Rajmohan, MD, UCI Health neurologist. “This trial may help us move one step closer to a future with available treatments that could potentially slow the progression of Huntington’s disease.”
The therapy, hNSC-01, uses pluripotent neural stem cells derived from embryonic stem cells, which were manufactured through the UC Davis GMP facility. In animal studies, the cells have been shown to protect existing brain cells, replace lost cells, rebuild impaired brain circuits, release helpful proteins, such as brain-derived neurotrophic factor (BDNF), and reduce harmful protein accumulations that damage brain cells. The stem cells were also shown to be safe over long periods in mice.
The clinical trial will enroll 21 people ages 18 to 65 with early-stage Huntington’s disease. Twelve participants will be enrolled into a Phase Ib dose-escalation group and nine in a Phase IIa expansion group.
The stem cells are implanted during a roughly six-hour surgical procedure done under general anesthesia. While lying face down in an MRI scanner, the patient receives stem cells implanted directly into the striatum deep in the brain, using a purchased proprietary therapy-enabling platform for navigation and surgical delivery. Damage to the striatum, which is responsible for motor control, decision-making, motivation and more, causes Huntington’s disease symptoms. Subjects will be closely monitored for safety as well as preliminary signs of potential benefit.
The clinical trial is made possible by a $12 million grant from the California Institute of Regenerative Medicine (CIRM), and the trial is coordinated through the UC Irvine Alpha Clinic.
The post First-in-Human Stem Cell Therapy Trial for Huntington’s Disease Begins at UCI Health appeared first on GEN – Genetic Engineering and Biotechnology News.
STAT+: Eli Lilly dives into hair loss treatments with investment in AI startup Absci
The pharmaceutical giants behind the monumentally successful weight loss drugs Wegovy and Mounjaro have been teasing an expansion into other aesthetic fields like hair loss or skin care.
Now, one of them is making a move, investing in a small startup developing a medication to spur hair growth, and potentially also treat endometriosis.
On Wednesday, Absci announced that it raised $100 million from a group led by Eli Lilly. Lilly brought the lion’s share of the funding, handing over $40 million in exchange for equity in Absci, which is publicly traded on the Nasdaq.
STAT+: U.S. health spending rose sharply in 2025, thanks to GLP-1 use and more care
Americans are seeing their doctors, getting hospital procedures, and filling prescriptions more frequently than economists and budget experts anticipated. Weight loss drugs, in particular, have morphed into their own special category of spending and are pushing budgets across the country to their limits.
Combining an increased amount of care with the country’s high baseline of prices has resulted in the health care system taking up more of the economy, new data show — findings that again reflect people’s widespread discontent with how unaffordable health care has become.
The country spent $5.7 trillion on health care in 2025, a 7.3% increase from 2024, according to the latest government figures published in the journal Health Affairs on Wednesday. That amounted to almost $16,500 per person.
BIO 2026: CEO Calls for U.S. Biotech Urgency and International Competitiveness
SAN DIEGO — Biotechnology is entering one of the most transformative periods in its history. But, according to Biotechnology Innovation Organization (BIO) CEO John Crowley, outdated regulations, rising development costs, and global competition threaten to slow progress unless policymakers act.
At the 2026 BIO International Convention in San Diego this week—which drew “roughly 20,000 attendees,” according to the organizers—Crowley outlined a vision for the future of biotechnology centered on accelerating clinical research, embracing artificial intelligence, and maintaining U.S. leadership in a rapidly evolving global bioeconomy.
The grassroots gauntlet
Crowley’s personal journey as a father shaped his path into biotechnology. In the late 1990s, two of his children were diagnosed with a rare form of muscular dystrophy. He left Bristol-Myers Squibb’s marketing department to co-found a biotechnology company with an Oklahoma academic researcher over scientific progress.
The struggle to get funding was immense. Crowley reflected on his first BIO convention in 2000 amidst the excitement of the Human Genome Project: “I came and there were tens of thousands of people partnering as there is today, still a quarter of a century later. Being the 31-year-old CEO of a small startup in Oklahoma City with no money, literally nobody signed up to meet with me and nobody accepted my meeting request.”
Crowley recalled going to the main stage, where a gentleman, rendered quadriplegic through a horse accident, came out on the stage and said, “Biotechnology—it’s a great big word that just means hope. It’s my hope that someday I can hold my wife’s hand on the beach or throw a ball to my kids.”
Crowley, empty-handed, returned to Oklahoma City and was able to scrounge up the funds for his startup, Novazyme Pharmaceuticals, which was ultimately funded by home equity loans and credit card advances to develop rare disease treatments. Just one year later, Novazyme was acquired by Genzyme Corporation for $225 million.
The experience engrained in Crowley two main concepts: first, developing therapeutics doesn’t always start in big pharma but, rather, often has grassroots origins; second, and relatedly, it’s an almost impossible battle for anyone outside of big pharma to fight.
“That’s the way so much of our science happens,” Crowley said. “It comes out of great universities, and it’s a scientist and entrepreneur—and increasingly, families, patients, and patient advocates—leading the way and going through the whole journey, running that gauntlet of making medicines.”
Modernizing clinical trials and accessible AI
To achieve the vision of maximizing the development and reach of biotechnology, Crowley identified a handful of problems, including the need to change the current system of clinical trials. Crowley praised the FDA’s new “Project Trailblazer” initiative to modernize experimental therapy human testing. He argued that clinical trials have become excessively burdensome and costly, limiting innovation and delaying patient access to new treatments.
Over the past year, Crowley and BIO have worked with regulators and industry stakeholders to identify development bottlenecks. “The FDA needs to continue to be the gold standard of the world,” he said, while emphasizing that modernization is necessary to make the agency a stronger “beacon of innovation.” BIO has proposed several reforms, including measures designed to streamline trial approvals and improve the efficiency of regulatory review.
Describing recent discussions among BIO’s board of directors, which includes executives from both major pharmaceutical companies and small biotechnology startups, Crowley said there were two major strategic topics that emerged that dominated the conversation: China and AI.
For AI, the question wasn’t about whether it could revolutionize biotechnology; rather, it had to do with making AI capabilities accessible to companies of all sizes. Crowley noted a major disparity. “Our biggest companies have the resources and the focus to think about AI. They’ve got hundreds or more people focused on this. Our small companies don’t have those resources,” he said.
Crowly continued, “It’s also a challenge because in our industry we would work on such long timelines, and it’s hard for an entrepreneur and biotech of a small or a mid-sized company who’s invested years to get to…starting Phase III, and all of a sudden you’ve got this massive disruptive technology. That’s exactly what AI is going to be.”
The solution, according to Crowley, is for BIO to be at the forefront to enable the rapid implementation of AI into drug development paradigms, clinical trials, and the regulatory review process.
Challenging China
Crowley’s most stressed point was that the United States must remain competitive against growing international rivals, particularly China. “Drug development has just gotten too costly and burdensome, and it takes too much time,” said Crowley. In this [global] bioeconomy where we need to compete and outcompete countries like China, these are reforms that are needed.”
He characterized biotechnology as a matter of national security and argued that the United States should treat the industry as a strategic asset. While supporting bipartisan efforts in Washington to strengthen domestic biotechnology capabilities, he cautioned against policies that could create unintended consequences or limit access to potentially life-saving technologies.
“The world is a better, safer, healthier, and more prosperous place when the United States and its allies continue to lead in biotechnology,” Crowley said.
China has identified biotechnology as a strategic priority through multiple national development plans and has invested heavily in scientific infrastructure, manufacturing capacity, and research capabilities. Crowley argued that the most effective response is not isolation but improving the competitiveness of the U.S. innovation ecosystem.
Crowley repeatedly returned to what he described as “man-made problems” holding the industry back. While scientific challenges will always exist, Crowley said barriers such as complex regulations, insufficient research funding, delays in patient access, and rising out-of-pocket healthcare costs are obstacles that policymakers can address. “We can’t come to this convention and cure every cancer,” he said. “But if we get together with policymakers and lawmakers, we can pretty quickly solve a lot of these man-made problems if we have the will.”
50 years down, 50 years ahead
As biotechnology celebrates more than 50 years of innovation, Crowley argued that the industry’s future will depend not only on scientific breakthroughs but also on its ability to modernize the systems that govern how those breakthroughs reach patients.
“I hope you see, when you’re here at this convention, that it captures that entrepreneurial spirit,” said Crowley. “It has to be grounded in great science and research, and it’s an exciting time to be in biotech, not just reflecting about all our successes and our many failures and challenges along the way in 50 years and looking out in the months, years, and next 50 years about what biotechnology can do to extend and enhance life and to alleviate an enormous amount of human suffering.”
With advances in gene editing, genomic medicine, artificial intelligence, and cell therapies accelerating simultaneously, Crowley believes the next era of biotechnology could surpass anything seen before—provided the industry can remove the barriers standing in its way.
The post BIO 2026: CEO Calls for U.S. Biotech Urgency and International Competitiveness appeared first on GEN – Genetic Engineering and Biotechnology News.
AI-CURA Automates Genetic Variant Classification
A new AI framework can classify hundreds of genetic variants as accurately as a human expert in a fraction of the time, research suggests.
Combining AI-assisted CURAtor (AI-CURA) with the latest large language models (LLMs) could streamline the diagnosis of rare genetic diseases.
The workflow system, described in Science Translational Medicine, performed as well as clinical experts in classifying 150 variants, while adhering to complex expert guidelines.
It was also able to categorize 150 variants with conflicting classifications.
“This study pushes the boundaries of fully automated variant interpretation,” commented senior journal editor Catherine Charneski, PhD, from the University of Bath.
Whole genome sequencing (WGM) has proven pivotal in ending the prolonged diagnostic odyssey of many patients with rare genetic disorders.
To manage the huge number of variants identified through WGS, attempts have been made by expert associations and working groups to establish guidelines and recommendations.
These now form a widely adopted classification system that categorizes variant-associated evidence into distinct rule-based categories.
But while some rules can be readily automated, most evidence still needs manual interpretation of the literature. This requires variant curators to possess broad knowledge across various aspects of molecular biology and genetics, as well as a deep understanding of expert recommendations to accurately interpret and score variants.
Wei Ma, PhD, and colleagues from the Hong Kong Genome Project (HKGP) therefore developed AI-CURA, a fully automated framework for variant classification that integrates LLMs to handle both literature-independent and literature-dependent evidence.
The tool integrates the assessment of evidence for non–literature-based criteria, which can be automated using standard bioinformatic tools, with a separate LLM-supported assessment of literature-based evidence.
Two state-of-the-art LLMs—DeepSeek-R1 and o3-mini-high—were tested for their ability to summarize literature-derived evidence relevant to variant classification.
The team found that the open-source DeepSeek-R1 outperformed o3-minihigh and had high sensitivity and 100% specificity in interpreting rules from the American College of Medical Genetics and Genomics (ACMG) that require understanding literature-based evidence.
They then tested it using 150 variants curated by ClinGen experts, with 150 expert-curated variants and 150 variants with conflicting classifications from the Clinical Genome Resource.
The open-source LLM DeepSeek-R1 showed high concordance with ClinGen experts in establishing a final diagnosis.
“In this study, DeepSeek-R1 demonstrated high accuracy (89.3 to 100%) in determining the application of seven literature-dependent ACMG rules,” the authors reported.
They added: “Our use of LLMs substantially streamlined the variant analysis and interpretation process. LLMs can finish summarizing the literature evidence in minutes.
“In comparison, curators in the HKGP typically spend around four hours per patient on WGS curation, with most of this time dedicated to reviewing literature.”
The post AI-CURA Automates Genetic Variant Classification appeared first on Inside Precision Medicine.
Light Sensor Detects Ultra-Low Levels of Traumatic Brain Injury Biomarkers
Researchers in China have developed a biosensor chip that uses light to detect extremely low concentrations of biomarkers of traumatic brain injury (TBI) at concentrations as low as femtograms per milliliter. The technology could one day be used to make faster diagnoses after a head injury, helping doctors choose the best treatment course and providing early warning of complications.
“Although several biomarkers have been validated as indicators of TBI, current methods for measuring them are time-consuming and require multiple complex laboratory steps,” said Guangyuan Li, PhD, professor at the Beijing Institute of Technology. “To address this challenge, we developed metasurface biosensors that are exceptionally sensitive, allowing them to produce clear, reliable optical signals even when only tiny amounts of a biomarker are present.”
The biosensor achieves its sensitivity thanks to metasurfaces—ultra-thin materials with microscopic patterns etched on them, which enable the device to manipulate light very precisely. For this study, Li and colleagues coated a gold metasurface with antibodies that specifically target TBI biomarkers. When the target molecules bind to the antibodies on the metasurface, the light wavelengths it reflects change slightly, indicating the presence of the biomarker even at extremely low concentrations.
To test this approach, the researchers built two separate sensors targeting two key biomarkers of TBI: the glial fibrillary acidic protein (GFAP) and S100 calcium-binding protein β (S100β). Results showed that the sensor could accurately detect subtle wavelength shifts depending on the biomarker concentration, with a sensitivity as low as under a femtogram per milliliter. This response was highly sensitive to the target biomarker, even when other biomarkers were present in the sample.
In recent years, light-based sensors have been increasingly gaining traction as diagnostic tools thanks to their potential to make biomarker detection much more precise compared to conventional methods, with promising applications currently being explored in early cancer diagnosis and real-time monitoring of diabetes.
However, more work will be needed before this technology can be routinely used in a clinical setting. With further development, the platform could be adapted to create metasurface sensors capable of detecting multiple biomarkers simultaneously to offer a more complete picture of a patient’s state in a short period of time. Going forward, Li and colleagues plan to continue working to reduce the costs of manufacturing the sensor, adapting fluid handling and packaging for clinical use, and ultimately validating the technology in clinical trials to assess its performance in a real-world setting.
“If developed into a point‑of‑care format, this technology could help provide faster and accurate answers after brain injury—perhaps using just a finger prick,” said Yunhui Liu, PhD, associate professor at the Shenzhen Institutes of Advanced Technology. “This could potentially reduce unnecessary CT scans for low‑risk cases while flagging higher‑risk patients earlier. It could also enable more accessible biomarker detection in ambulances, rural clinics, sports settings or emergency departments where time matters.”
The post Light Sensor Detects Ultra-Low Levels of Traumatic Brain Injury Biomarkers appeared first on Inside Precision Medicine.
Social Determinants of Health Improve Disease Risk Prediction Beyond Genetics Alone
A study by researchers at the Icahn School of Medicine at Mount Sinai has found that social determinants of health—including environmental conditions, health behaviors, access to resources, and social well-being—can contribute as much as or more than genetic risk in predicting several common diseases. The research, published in The American Journal of Human Genetics, showed that incorporating social, behavioral, and environmental information into disease-risk models improved prediction when incorporated with genetic information for conditions including asthma, chronic kidney disease, coronary heart disease, high cholesterol, breast cancer, and prostate cancer.
“Genes are an important part of the equation, but they do not determine destiny,” said senior author Samira Asgari, PhD, an assistant professor of genetics and genomic sciences at Mount Sinai. “We found that the circumstances of people’s lives—their environments, behaviors, and social experiences—can contribute as much as genetics to predicting disease risk. To truly understand health, we have to look at the whole person, not just their DNA.”
According to the researchers, complex diseases arise through the interaction of genetic predisposition with environmental, behavioral, and social influences, yet these factors are often studied in isolation. Existing genetic models often rely on polygenic risk scores, while epidemiological approaches focus on lifestyle, environmental exposures, or social factors independently. The researchers sought to bridge that gap by integrating both types of data into a single risk prediction framework.
To conduct the study, the team analyzed data from 413,457 participants in the All of Us Research Program, a nationwide research effort in the U.S. supported by the National Institutes of Health. The team combined genetic information, electronic health records, and survey responses, with more than 100 environmental, behavioral, and social variables were evaluated, to create a broad picture of the different factors that may influence health.
Rather than selecting a limited number of known social risk factors in advance for their survey, the researchers used a statistical technique called multiple correspondence analysis, or MCA. The approach converted more than 100 categorical social, environmental, and behavioral measures into low-dimensional representations that helped identify patterns of non-genetic risk.
The choice to use MCA distinguished the study from many previous approaches. Past methods often have depended on selecting a small set of established risk factors or using statistical procedures that prioritize only the strongest predictors. By contrast, MCA identifies patterns across many correlated variables simultaneously, allowing researchers to examine broader social and environmental variables without assuming beforehand which factors have the most influence on health.
The analysis found known contributors to disease risk such as economic status and smoking, but also identified factors that receive less attention in published studies, including loneliness and spirituality. First author Abhijith Biji, a PhD candidate at Icahn School of Medicine, said the data showing associations involving loneliness was particularly notable.
“Some risk factors, such as smoking, have been studied extensively for decades,” Biji said. “What is especially intriguing is that we also observed associations involving factors like loneliness. Understanding how these experiences may become biologically embedded could open new avenues for research and ultimately improve our understanding of disease.”
When the researchers incorporated the MCA-derived measures into prediction models alongside demographic information and polygenic risk scores, predictive performance improved across all six diseases studied. For four of the six diseases, the gains from the MCA-based measures exceeded those attributable to polygenic risk scores.
The findings also suggested that genetic and non-genetic influences generally act independently rather than modifying one another. The researchers found little evidence for broad gene-environment interactions. Instead, inherited genetic risk and social, behavioral, and environmental context appeared to contribute additively to disease risk.
“This additive relationship suggests that interventions targeting social and behavioral factors can reduce disease risk regardless of genetic background, offering hope for broadly applicable public-health strategies,” the researchers wrote.
The researchers noted that the study does not establish causation. Because many survey responses were collected at a single point in time and some exposures may have occurred after disease onset, the findings should be considered as contributions to disease-risk prediction rather than proof that specific factors cause disease.
Building on this work, the team next will seek to integrate social determinants of health with additional biological measures and look mechanisms that may directly connect social experiences to disease. The investigators will also bring in longitudinal data, harmonize survey instruments across cohorts, and integrating other data types to better understand how environmental, behavioral, and social factors influence disease development and interact with biological processes.
“Our goal is to build a more complete understanding of health and disease,” Asgari said. “By combining genetics with social and environmental context, we can move toward risk models that better reflect the realities of people’s lives and help advance more personalized approaches to health.”
The post Social Determinants of Health Improve Disease Risk Prediction Beyond Genetics Alone appeared first on Inside Precision Medicine.

