Researchers find AI models can score well on accuracy measures while missing key signs of mental health concerns; they introduce Text Psychometrics, a new framework for testing these tools more rigorously.
New York, NY — Today the Child Mind Institute, in collaboration with Harvard University and the Massachusetts Institute of Technology, released a new research paper that found that AI tools cannot yet provide valid psychological assessments and require a more rigorous testing process. The paper reaffirms that mental health professionals are still best suited to assess and address the needs of people seeking mental health resources. Additionally, researchers introduced a framework for evaluating whether AI systems can measure mental health in a scientifically sound, reliable, and fair way for future development. The paper is in press at PLOS Digital Health.
Every day, people describe their inner lives in text — including crisis hotline messages, social media posts, therapy transcripts, journal entries, and more. As AI becomes increasingly capable of analyzing language, researchers are beginning to use these tools as a new way to conduct psychological assessments.
However, unlike traditional questionnaires used by psychologists, most AI language tools are insufficiently tested. Experts have spent a century building rigorous statistical standards for questionnaires within the field of psychometrics. Does a test measure what it claims? Does it give consistent answers? And does it work equally well for everyone? AI language tools are rarely held to that same standard.
In this paper, researchers introduce the concept of Text Psychometrics, a framework for evaluating whether AI and natural language processing (NLP) tools can measure psychological constructs in text in a scientifically sound way. Rather than asking only whether a model can match human judgments, the co-authors argue that researchers should also examine whether these tools capture the full range of how people express symptoms, remain reliable across different psychological situations, and work fairly across different patient populations.
“This research reaffirms that AI cannot replace mental health professionals to adequately assess and meet the needs of patients and people seeking mental health resources. An AI model may appear accurate while still missing important ways people express psychological symptoms,” says Daniel Low, PhD, a research scientist at the Child Mind Institute and the study’s lead author. “AI can help researchers and clinicians analyze language at scale and find patterns that would otherwise be very challenging, but these AI models do not currently understand the full approach that experienced clinicians take when making assessments. With our new framework, our goal is not to replace human expertise, but to build tools that are helpful while being more transparent, trustworthy, and scientifically grounded to meet the needs of patients.”
To develop the paper and framework, researchers analyzed thousands of crisis counseling conversations and public social media posts. They compared several approaches for identifying mental health concerns, including large language models and other language-analysis methods. The team found that some AI systems performed well, on average, with commonly used accuracy measures. But they still failed to recognize many clear and important expressions of mental health concerns because they were less common and therefore did not affect the general accuracy. In other words, a model could appear successful while overlooking language that a clinician or researcher would consider highly relevant.
To address this issue, the researchers developed a new framework to evaluate language models that includes different types of validation, including what they call “content validity test sets.” Rather than asking only whether a model agrees with human judgments, the approach examines whether the model recognizes the full range of ways people express a psychological symptom. For example, to screen for eating disorders, a model would need to not only capture mentions of anorexia and bulimia but also purging, chewing and spitting, and other symptoms.
“A mental health model should not be considered trustworthy simply because it performs well on a single metric,” Dr. Low explains. “Researchers should also ask whether it captures the most important expressions of a symptom, performs consistently, protects privacy, and works fairly across different groups of people, among many other desirable properties we describe.”
In a second study, the team examined whether language from the beginning of a crisis counseling conversation could predict who would later be sent an emergency service intervention towards the end of the conversation. The researchers found that language patterns associated with suicide risk could provide useful signals, while also revealing which factors out of about 50 were most strongly linked to acute risk. Mentions of lethal means for suicide, substance use and overdose, and having the texter write more than the counselor were the most predictive variables for detecting the need for an emergency service intervention.
Human clinicians consider far more than written words when evaluating a person’s mental health, including clinical history, life circumstances, nonverbal cues — if available — and professional judgment. The findings suggest that AI systems are not yet at this level.
The researchers hope Text Psychometrics will help consolidate efforts to improve how AI-based mental health assessments are developed and evaluated. By applying the same rigorous standards used for traditional psychological tests, they believe language-based AI tools can become more reliable, interpretable, and useful for research, clinical care, and public health.
The study was authored by Daniel Low, PhD, Patrick Mair, PhD, and Matthew K. Nock, PhD, of Harvard University, and Satrajit S. Ghosh, PhD, of the Massachusetts Institute of Technology and Harvard Medical School.
The research was supported by training grants from the National Institute of Mental Health and the National Institute on Deafness and Other Communication Disorders. It also received support from the RallyPoint Fellowship, Amelia Peabody Professional Development Award, Chet and Will Griswold Fund, Tommy Fuss Collaborative STAR Fund, and Lann and Christopher Woehrle Psychiatric Research Fund. The funders had no role in the study’s design, data collection and analysis, publication decision, or manuscript preparation.
The authors thank Crisis Text Line for providing access to de-identified data used in this research. Crisis Text Line did not influence the study design, analysis, interpretation of findings, or conclusions.
Learn More About Our Work
Learn more about the AI, Risk, and Contemplative Science (ARC) Lab and the Center for Data Analytics, Innovation, and Rigor (DAIR) at the Child Mind Institute.
Crisis Resources
If you or someone you know is struggling or in crisis:
Call or text 988 anytime in the United States to reach the Suicide & Crisis Lifeline
To reach Crisis Text Line, text HOME or HOLA to 741741 to connect with a live, trained volunteer crisis counselor.
About the Child Mind Institute
The Child Mind Institute is dedicated to transforming the lives of children and families struggling with mental health and learning disorders by giving them the help they need. We’ve become the leading independent nonprofit in children’s mental health by providing gold-standard, evidence-based care, delivering educational resources to millions of families each year, training educators in underserved communities, and developing tomorrow’s breakthrough treatments.
The post New Research Calls for Stronger Standards When Using Large Language Models and Other AI Tools for Psychological Assessment appeared first on Child Mind Institute.
