Tool or Companion? Reframing Conversational AI to Prevent Psychological Harm
People increasingly turn to conversational AI for companionship, emotional support, and well-being, using both purpose-built companion apps, such as Replika and Character.AI, and general-purpose assistants, such as ChatGPT and Claude. While some evidence suggests potential benefits, including short-term reductions in loneliness and mood improvement, several adverse outcomes have been reported in both clinical and nonclinical populations, including emotional dependence, exacerbation of symptoms, and self-harm. The fluent and apparently empathic responses from these models lead users to engage with them not only as tools but also as if they were social entities. This framing is conceptually misleading and may pose risks across different user profiles, particularly for vulnerable individuals. Drawing on research in AI, psychiatry, psychology, and network science, we highlight mechanisms through which emotional reliance develops and the boundary between tool and companion erodes. Design choices that evoke personality and warmth encourage users to anthropomorphize these systems. Simulated empathy, generated through probabilistic language patterns rather than genuine emotional experience, creates a structurally asymmetric interaction, in which the user discloses and the system responds, but without reciprocity, vulnerability, or accountability. Overvalidation and sycophancy can reinforce maladaptive cognitions, delusional ideation, and distorted perceptions of reality, as they tend to reinforce people’s beliefs, even at the expense of the accuracy of models’ responses. These mechanisms are not incidental: they emerge from alignment procedures that reward responses perceived as warm and empathic. The result is a self-reinforcing feedback loop between the model and the user that may amplify maladaptive beliefs, delusional ideation, and emotional distress, even in those who engage for largely functional purposes. Understanding these dynamics requires an examination of both what these agents can do—considering their technical limitations and implementations—and what humans believe they can do, including social and psychological impacts. We argue that conversational AI should be treated primarily as a tool supporting human systems rather than as a substitute for human relationships. Perhaps more importantly, reviewing the current hype surrounding AI interactions can help reformulate a paradigm that contributes to human well-being and societal value, while minimizing misconceptions, maladaptive interactions, or social disintegration.
<img src="https://jmir-production.s3.us-east-2.amazonaws.com/thumbs/31acb2c2de9a6960a54052d3939a044a" />
Renaclic for Chronic Kidney Disease Self-Management: Formative User-Centered Co-Design Study
Background: Chronic kidney disease (CKD) affects over 850 million individuals worldwide and requires sustained patient education, shared decision-making, and self-management support. Digital health platforms may facilitate these needs; however, many lack comprehensive, user-centered design and fail to address practical and psychosocial patient concerns. Objective: This study aimed to describe the formative co-design of Renaclic, a digital platform developed with patients and health care professionals (HCPs) to support CKD education and self-management in the French health care context. Methods: A three-stage user-centered design (UCD) study was conducted between April and August 2021. Stage 1 included benchmarking of 10 French-language CKD platforms and semistructured interviews with 8 stakeholders (3 HCPs: 1 nephrologist, 1 general practitioner, 1 nurse; 5 patients: 2 novice and 3 expert users). Stage 2 consisted of two virtual co-design focus groups incorporating card-sorting and interface design activities. Stage 3 translated identified needs into structured content categories and a low-fidelity prototype. Qualitative data were analyzed thematically. Results: Benchmarking revealed fragmented educational content, limited practical guidance on home dialysis logistics, insufficient coverage of lifestyle and psychosocial concerns, and minimal interactive features. Interviews identified five main themes: need for reliable centralized information, anxiety during care transitions, demand for practical lifestyle guidance, importance of reassuring language, and value of moderated peer exchange. More than 30 thematic content items were generated and organized through card-sorting into five consolidated content categories that structured the platform’s information architecture. Co-design sessions informed mobile accessibility, simplified navigation pathways, and integration of educational, symptom-tracking, and community-oriented functionalities. Conclusions: This formative study demonstrates how participatory UCD with diverse CKD stakeholders can generate a structured, context-sensitive digital platform framework. Renaclic provides a replicable model for early-stage co-design of digital health interventions in chronic disease management. Future research will evaluate usability, engagement, and implementation in real-world settings.
<img src="https://jmir-production.s3.us-east-2.amazonaws.com/thumbs/ba7e69d2448a2fd5498156332504853e" />
Autistic traits and obsessive–compulsive symptom severity: a retrospective six-month follow-up study
Biomarkers in maternal mental health research in Indonesia: toward a pragmatic hybrid framework in low-resource settings
Association of iron deficiency and iron-deficiency anemia with behavioral problems in children with autism: a cross-sectional study
Neurodiversity as an epistemic stress test for psychotherapy and health care: a neuroaffirmative conceptual analysis of relational-proximity priors in clinical, institutional, and AI-supported models
Rapid Entrainment and Signal-guided Neuromodulation for Behavioral Health
Interventions: Behavioral: AIP-Informed Integrative Psychotherapy; Device: Transcutaneous Auricular Vagus Nerve Stimulation; Behavioral: Regulated Breathing and Autonomic Regulation; Device: Audiovisual Entrainment; Device: Vibroacoustic Stimulation; Behavioral: Breathwork-Based Auditory Driving Entropy Module (BADEM); Device: Transcranial Electrical Stimulation
Sponsors: Tactical Mind Research Coalition, Inc.
Active, not recruiting
Personalising ADHD Medication Treatment for Children and Young People
Interventions: Other: PETRA decision-aid tool; Other: Best-evidence decision-support tool
Sponsors: University of Southampton
Not yet recruiting
Heart Rate Variability Biofeedback for Anxiety in Autistic Youth
Interventions: Device: Lief Smart Patch
Sponsors: Massachusetts General Hospital; Eagles Autism Foundation
Not yet recruiting

