Category: Imported News

Category Added in a WPeMatico Campaign

Posted on

A 2-Tiered Rescue Protocol to Mitigate Difficulty-Based Failures of ChatGPT (GPT-5) and Gemini on the German M2 Medical Examination: Evaluation Study

Background: Large language models (LLMs) have demonstrated expert-level performance on medical licensing examinations, but most benchmarks focus on final accuracy. Critical gaps remain in understanding model efficiency (latency), the efficacy of tiered “rescue” protocols for error correction, and the systematic correlation between performance and human-rated question difficulty. The German M2 examination, paired with the AMBOSS platform’s user data–driven difficulty ratings, provides an opportunity to map AI performance against human cognitive load. Objective: This study aimed to move beyond singular accuracy scores by (1) evaluating and comparing the baseline (tier 1; T1) accuracy and response latency of next-generation rapid-response LLMs, (2) analyzing the efficacy of a 2-tiered rescue (tier 2; T2) protocol in correcting initial errors, and (3) correlating model performance with the user data–driven AMBOSS difficulty rating. Methods: We evaluated 4 LLMs (Gemini 2.5 Flash, Gemini 2.5 Pro, GPT-5 Instant, and GPT-5 Thinking) on the complete 316-item German M2 (Fall 2024) medical examination, including all multimodal (image-based) questions. A zero-shot copy-paste prompting strategy was used, and outputs were evaluated against ground-truth answers using a strict exact-match criterion. A 2-tiered protocol was used: T1 (Gemini Flash and GPT-5 Instant) provided baseline responses. If incorrect, a T2 (Gemini Pro and GPT-5 Thinking) model was deployed as a “rescue.” Performance was analyzed using the McNemar test, the Wilcoxon signed-rank test, the Fisher exact test, and logistic regression. Results: Baseline (T1) accuracy was identical at 91.5% (289/316; 95% CI 87.85%‐94.06%) for both Gemini 2.5 Flash and GPT-5 Instant, with 27 errors each. However, Gemini Flash (mean 1.57, SD 1.06 s) was significantly faster than GPT-5 Instant (mean 2.07, SD 1.89 s; <.001). Additionally, GPT-5 Instant expended significantly more time on incorrect answers compared with correct ones (=.002), whereas Gemini Flash showed no such hesitation (=.81). The T2 rescue rate for GPT-5 Thinking (13/27, 48.2%; 95% CI 30.74%‐66.01%) was higher, though not statistically significant (=.41), than that for Gemini 2.5 Pro (9/27, 33.3%; 95% CI 18.64%‐52.18%). This rescue protocol elevated final accuracy to 94.3% (298/316; 95% CI 91.18%‐96.37%) for the Gemini system and 95.6% (302/316; 95% CI 92.70%‐97.34%) for the GPT-5 system (=.48). A strong, inverse relationship with difficulty was found: for every 1-point increase in difficulty, the odds of a correct T1 response decreased by 42.1% (odds ratio 0.579, 95% CI 0.425‐0.788; <.001) for Gemini Flash and 47.7% (odds ratio 0.523, 95% CI 0.379‐0.720; <.001) for GPT-5 Instant. This negative correlation persisted even after the rescue (=.01 and =.006, respectively). Conclusions: Expert-level LLM performance on the German M2 examination masks a critical vulnerability: a decrease in accuracy correlated with increased question difficulty. A 2-tiered “rescue” system is an effective strategy to mitigate these difficulty-based failures and achieve >95% accuracy.
Posted on

Development and Usability Evaluation of an E-Learning Tool for Blended Learning in Pediatric Endocrinology: Formative Pilot Study

Background: Residents in pediatric endocrinology subspecialty units encounter diverse educational scenarios spanning theory, skills, and attitudes; yet, brief residencies frequently limit their exposure to certain clinical cases. Research in medical education demonstrates that e-learning can address such challenges efficiently. We implemented a blended learning model grounded in the Kolb learning cycle that uses structured, case-based e-learning. Objective: We aimed to evaluate the utility and usability of blended learning using a novel e-learning tool. Methods: We used a problem-solving approach and used the physical separation of case-based e-learning (interactive, patient scenario–based online modules) and theoretical content delivery as the educational model for residents in a pediatric endocrinology and diabetology unit. Residents worked asynchronously (on their own time, not simultaneously with others) on clinical scenarios and completed formative assessments (practice tests designed to provide feedback for learning rather than grades) with immediate feedback using a flipped classroom teaching method, in which students review material before group instruction. In addition, all cases could be discussed with specialists during face-to-face learning opportunities through a blended learning approach that combines online and in-person elements. We evaluated Kirkpatrick level 1 (reaction, how participants respond to training) and level 2 (learning, measured as an increase in knowledge or capability) outcomes using the postgraduate Medical E-learning Evaluation Survey (MEES) and the User Experience Questionnaire (UEQ), which assesses users’ perceptions of e-learning platforms. Results: Questionnaires from 12 pediatric residents and 1 questionnaire from a fourth-year medical student were evaluated. The main strengths identified were the tool’s support for applying content to daily clinical work (12/13, 92% users), provision of timely summaries (n=9, 69% users), access to reliable information sources (n=9, 69% users), and immediate feedback on responses (n=8, 62% users). Key weaknesses included device compatibility for e-learning (n=5, 38% users), limited content personalization (n=4, 31% users), and a lack of a navigation aid (n=4, 31% users). No significant functional issues were reported. The UEQ evaluation showed that dependability received the lowest rating, while attractiveness and stimulation received the highest rating. Conclusions: Our e-learning proposal provides a practical way to apply theoretical knowledge through interactive clinical cases. Evaluations show that users are highly motivated to engage with e-learning, highlighting our tool’s adaptability and effectiveness for postgraduate medical education in pediatric endocrinology. Identifying strengths and weaknesses will guide future improvements. Evaluating various aspects of e-learning remains crucial, as these aspects can affect learning outcomes. However, more longitudinal evaluations of e-learning are necessary to achieve a comprehensive understanding of its effectiveness.
<img src="https://jmir-production.s3.us-east-2.amazonaws.com/thumbs/6e71bb0793753160fbbcd61e16945cfc" />

Posted on

Digital Reassurance Seeking in OCD

– by Jordan Karr, PhD

We all need reassurance once in a while. Receiving reassurance from a friend, loved one, or expert can reduce short-term distress, foster a sense of connection, and help us move forward in a productive way. On the other hand, excessive reassurance seeking can lead to a painful cycle of stress and doubt. For individuals with obsessive-compulsive disorder (OCD), reassurance is fleeting and is quickly followed by a resurgence of perceived threat and a compelling drive to seek additional reassurance.

As an OCD and anxiety specialist, I am no stranger to reassurance traps. I see them in the patient who repeatedly consults doctors but never fully trusts their answers; the partner who keeps asking, “Do you still love me?” while scrutinizing every response; and the teenager who needs her homework checked repeatedly but does not learn to tolerate uncertainty. While the content of excessive reassurance seeking may vary, its function remains the same, relieving anxiety momentarily while allowing obsessions to rebound more strongly in the long run. In each example, we see an individual desperately trying to protect something they hold dear while being misled by one of OCD’s most convincing lines: “What if this time the reassurance sticks?”

Psychologists have long argued that excessive reassurance seeking helps maintain OCD, anxiety, and depression (Abramowitz et al., 2002; Burns et al., 2006). What is new is the landscape. Today, reassurance is available instantly and endlessly through search engines, medical websites, social media, and artificial intelligence tools. Emerging research suggests that online reassurance seeking may be as prevalent as interpersonal reassurance seeking (Parsons & Alden, 2022). These digital reassurance traps present in a variety of forms, including:

  • Excessive symptom-checking on WebMD
  • Checking a loved one’s location on Find My Friends to make sure they are still alive or where they promised they would be
  • Asking Google Gemini whether an intrusive thought makes you a monster, a pedophile, or a bad Christian, Muslim, or Jew
  • Analyzing the contents of a partner’s Instagram to see whether their feelings have changed
  • Endlessly searching Reddit for answers to existential questions
  • Obsessively checking likes and read receipts to monitor the durability of a relationship
  • Finding creative ways to ask ChatGPT whether you might be responsible for spreading a virus or causing someone harm
  • Compulsively checking the news to confirm whether a feared event has taken place

Why do some individuals seek reassurance primarily from others in face-to-face contexts while others turn to the internet? Findings from one study suggest that individuals may be more likely to seek interpersonal reassurance when they desire emotional support, whereas online reassurance seeking is more likely when individuals feel ashamed or fear judgment from others (Parsons et al., 2025). In the same study, shame was reported more frequently among individuals with OCD. The perceived anonymity of the internet creates a compelling environment for seeking reassurance about our most distressing fears and the fears we feel ashamed to have.

Wait, hold on… shouldn’t having access to all the information on the internet be empowering? After all, there’s that famous saying, “Knowledge is Power.” Well I’m not sure what Francis Bacon would say if he could scroll on TikTok, but in the age of the internet, more information does not always mean more power or knowledge. In fact, information overload seems to trigger excessive reassurance seeking online (Yang & Luo, 2024). Once digital algorithms detect health-related concerns, users may be exposed to increasingly frequent and targeted content, further amplifying health anxiety and driving additional reassurance seeking online (Zhang et al., 2024).

Despite these challenges, evidence-based treatments offer hope. Exposure and response prevention (ERP), the gold-standard treatment for OCD, helps individuals tolerate uncertainty and break free from compulsive behaviors. ERP therapists are prepared to respond to clients who compulsively seek reassurance while taking care to avoid reinforcing a client’s anxiety and setting boundaries when appropriate. On the other hand, AI chatbots are available 24/7 and will continue to reassure users when it would be clear to a skilled therapist that more reassurance is harmful. Some helpful activities I have encouraged clients to try include:

  • Going for a walk and leaving your phone behind
  • When you have the urge to seek reassurance, use a timer to delay. Start with 5 minutes and gradually increase the time as you go.
  • Imaginal exposure: Writing out OCD’s worst case scenario while refraining from seeking reassurance.
  • Embracing the uncertainty: When you have the urge to seek reassurance, lean into the uncertainty by responding “maybe that will happen, maybe not.”
  • Watch a movie that triggers obsessions but leave your phone in the other room
  • Try a fast from social media. Start with a short fast and increase the length of each fast as you gain confidence.
  • Agreeing with the obsessions: Whatever your OCD throws at you, respond by saying “Sure, that is true!”

Clinicians working with youth should also attend to parental accommodations that inadvertently facilitate online reassurance seeking. Supportive Parenting for Anxious Childhood Emotions (SPACE) is an evidence-based, parent-focused intervention that emphasizes increasing supportive statements while reducing unhelpful accommodations. Setting reasonable limits on smartphone use while responding with empathy and confidence can help youth build resilience and independence (i.e. “I know it’s tough to be away from your phone and I know you got this!”).

Finally, because shame plays a central role in digital reassurance traps, incorporating self-compassion practices may be beneficial. Compassion-focused exercises help individuals respond to their struggles with kindness rather than self-criticism, complementing ERP by fostering emotional resilience. I sometimes ask clients to imagine that a close friend or loved one is feeling ashamed because they are experiencing obsessions and are stuck in an OCD loop. Then, I invite them to write down how they would support this friend while paying special attention to how compassion feels in their body. By practicing compassion for others, we strengthen the same muscles in our brains that help us turn compassion inward.

If you are getting stuck in digital reassurance traps, you are not alone! Many of these technologies are brand new and we are learning how to integrate them into our lives in a healthy way. If you need help managing compulsive online habits, finding a therapist trained in ERP could be a useful step!

References

Abramowitz, J. S., Schwartz, S. A., & Whiteside, S. P. (2002). A contemporary conceptual model of hypochondriasis. Mayo Clinic Proceedings, 77(12), 1323–1330. https://doi.org/10.4065/77.12.1323

Burns, A. B., Brown, J. S., Plant, E. A., Sachs-Ericsson, N., & Joiner, T. E., Jr. (2006). On the specific depressotypic nature of excessive reassurance-seeking. Personality and Individual Differences, 40(1), 135–145. https://doi.org/10.1016/j.paid.2005.05.019

Parsons, C. A., & Alden, L. E. (2022). Online reassurance-seeking and relationships with obsessive-compulsive symptoms, shame, and fear of self. Journal of Obsessive-Compulsive and Related Disorders, 33, 100714. https://doi.org/10.1016/j.jocrd.2022.100714

Parsons, C. A., Kim, H. J., Singh, S., Lkhagva, T., Wang, J., & Alden, L. E. (2025). Covert or connected: Motivations for online and interpersonal reassurance-seeking in OCD. Journal of Anxiety Disorders, 115, 103057. https://doi.org/10.1016/j.janxdis.2025.103057

Yang, X., & Luo, X. (2024). Unpacking cyberchondria: The roles of online health information seeking, health information overload, and health misperceptions. Telematics and Informatics, 97, 102225.

Zhang, X., Zheng, H., Zeng, Y., Zou, J., & Zhao, L. (2024). Exploring how health-related advertising interference contributes to the development of cyberchondria: A stressor–strain–outcome approach. BMC Public Health, 24, 534.

Jordan Karr, PhD, is the owner of River Falls Therapy in Portland, OR. He is a licensed psychologist in Oregon and Virginia, and specializes in evidence-based therapies for OCD and anxiety disorders. Dr. Karr has experience working in outpatient, community-based, and school-based settings.

The post Digital Reassurance Seeking in OCD appeared first on International OCD Foundation.

Posted on
<![CDATA[Meta-analysis ranks add-on atypical antipsychotics for MDD, comparing symptom response and dropout risks to guide smarter next-step treatment.]]>