Tag: Healthcare provider

<strong>Background:</strong> Postoperative follow-up after brain tumor surgery is typically limited to intermittent clinic visits, leaving subtle neurological or general deterioration between visits underrecognized. Digital self-monitoring platforms may help fill this gap, but evidence in neuro-oncology is scarce, particularly regarding how patient-reported symptom trajectories can feed into future data-driven early warning systems. <strong>Objective:</strong> This study aimed to evaluate the feasibility, use patterns, and preliminary usability of a smartphone or web-based self-monitoring system for patients after brain tumor surgery and to explore simple rule-based digital alerts as a first step toward an advanced digital early warning framework. <strong>Methods:</strong> We conducted a single-center prospective pilot study including adults discharged after brain tumor surgery who had access to a smartphone and could use a web app. Participants completed brief symptom surveys consisting of 51 binary items across 7 symptom domains, with an automatically calculated daily total score and score history visualization. Feasibility was assessed by enrollment, retention, submission counts, and submission rates. A total of 4 interpretable alert rules based on current score, short-term worsening, new-onset symptom combinations, and persistence across domains were evaluated using each patient’s last 3 submissions as the analytic unit. Clinical deterioration was defined a priori as objective decline in performance status, new neurological deficit, radiologic progression, or clinically significant laboratory changes. Rule performance metrics and bootstrap CIs were computed. Usability and acceptability were evaluated using the System Usability Scale and additional adherence-related items. <strong>Results:</strong> Of 64 enrolled patients, 30 (47%) with ≥3 submissions formed the analysis cohort (median age 57, IQR 47.2–64.5 years; n=12.9, 43% malignant tumors); 6 (20%) experienced clinical deterioration during follow-up. Patients contributed a median of 8.5 submissions (mean 19.03, SD 30.12) at 1.7 surveys per week on average, indicating sustained but heterogeneous engagement. The best-performing rule, based on net short-term score increase, achieved an area under the receiver operating characteristic of 0.88, with sensitivity 0.83, specificity 0.92, and accuracy 0.90 on the last-window dataset, outperforming rules based solely on current score or multidomain persistence. Among 23 app users who completed the System Usability Scale, the mean score was 84.0, reflecting high perceived usability; higher-frequency users reported stronger perceived usefulness and habit-driven use. <strong>Conclusions:</strong> This pilot study demonstrates that a smartphone or web-based self-monitoring platform for patients with brain tumor is feasible and well accepted and that simple, transparent rules applied to longitudinal symptom scores show potential to capture early signals of clinical deterioration. However, given the small sample size, these predictive metrics are preliminary and require rigorous validation in larger, independent cohorts. These findings support further development of integrated digital early warning systems that combine patient-reported trajectories with clinical and physiological data to enhance postoperative neurosurgical care.

The availability of artificial intelligence for use in warfare is at the center of a legal battle between Anthropic and the Pentagon. This debate has become urgent, with AI playing a bigger role than ever before in the current conflict with Iran. AI is no longer just helping humans analyze intelligence. It is now an active player—generating targets in real time, controlling and coordinating missile interceptions, and guiding lethal swarms of autonomous drones.

Most of the public conversation regarding the use of AI-driven autonomous lethal weapons centers on how much humans should remain “in the loop.” Under the Pentagon’s current guidelines, human oversight supposedly provides accountability, context, and nuance while reducing the risk of hacking.

AI systems are opaque “black boxes”

But the debate over “humans in the loop” is a comforting distraction. The immediate danger is not that machines will act without human oversight; it is that human overseers have no idea what the machines are actually “thinking.” The Pentagon’s guidelines are fundamentally flawed because they rest on the dangerous assumption that humans understand how AI systems work.

Having studied intentions in the human brain for decades and in AI systems more recently, I can attest that state-of-the-art AI systems are essentially “black boxes.” We know the inputs and outputs, but the artificial “brain” processing them remains opaque. Even their creators cannot fully interpret them or understand how they work. And when AIs do provide reasons, they are not always trustworthy.

The illusion of human oversight in autonomous systems

In the debate over human oversight, a fundamental question is going unasked: Can we understand what an AI system intends to do before it acts?

Imagine an autonomous drone tasked with destroying an enemy munitions factory. The automated command and control system determines that the optimal target is a munitions storage building. It reports a 92% probability of mission success because secondary explosions of the munitions in the building will thoroughly destroy the facility. A human operator reviews the legitimate military objective, sees the high success rate, and approves the strike.

But what the operator does not know is that the AI system’s calculation included a hidden factor: Beyond devastating the munitions factory, the secondary explosions would also severely damage a nearby children’s hospital. The emergency response would then focus on the hospital, ensuring the factory burns down. To the AI, maximizing disruption in this way meets its given objective. But to a human, it is potentially committing a war crime by violating the rules regarding civilian life. 

Keeping a human in the loop may not provide the safeguard people imagine, because the human cannot know the AI’s intention before it acts. Advanced AI systems do not simply execute instructions; they interpret them. If operators fail to define their objectives carefully enough—a highly likely scenario in high-pressure situations—the “black box” system could be doing exactly what it was told and still not acting as humans intended.

This “intention gap” between AI systems and human operators is precisely why we hesitate to deploy frontier black-box AI in civilian health care or air traffic control, and why its integration into the workplace remains fraught—yet we are rushing to deploy it on the battlefield.

To make matters worse, if one side in a conflict deploys fully autonomous weapons, which operate at machine speed and scale, the pressure to remain competitive would push the other side to rely on such weapons too. This means the use of increasingly autonomous—and opaque—AI decision-making in war is only likely to grow.

The solution: Advance the science of AI intentions

The science of AI must comprise both building highly capable AI technology and understanding how this technology works. Huge advances have been made in developing and building more capable models, driven by record investments—forecast by Gartner to grow to around $2.5 trillion in 2026 alone. In contrast, the investment in understanding how the technology works has been minuscule.

We need a massive paradigm shift. Engineers are building increasingly capable systems. But understanding how these systems work is not just an engineering problem—it requires an interdisciplinary effort. We must build the tools to characterize, measure, and intervene in the intentions of AI agents before they act. We need to map the internal pathways of the neural networks that drive these agents so that we can build a true causal understanding of their decision-making, moving beyond merely observing inputs and outputs. 

A promising way forward is to combine techniques from mechanistic interpretability (breaking neural networks down into human-understandable components) with insights, tools, and models from the neuroscience of intentions. Another idea is to develop transparent, interpretable “auditor” AIs designed to monitor the behavior and emergent goals of more capable black-box systems in real time.  

Developing a better understanding of how AI functions will enable us to rely on AI systems for mission-critical applications. It will also make it easier to build more efficient, more capable, and safer systems.

Colleagues and I are exploring how ideas from neuroscience, cognitive science, and philosophy—fields that study how intentions arise in human decision-making—might help us understand the intentions of artificial systems. We must prioritize these kinds of interdisciplinary efforts, including collaborations between academia, government, and industry.

However, we need more than just academic exploration. The tech industry—and the philanthropists funding AI alignment, which strives to encode human values and goals into these models—must direct substantial investments toward interdisciplinary interpretability research. Furthermore, as the Pentagon pursues increasingly autonomous systems, Congress must mandate rigorous testing of AI systems’ intentions, not just their performance.

Until we achieve that, human oversight over AI may be more illusion than safeguard.

Uri Maoz is a cognitive and computational neuroscientist specializing in how the brain transforms intentions into actions. A professor at Chapman University with appointments at UCLA and Caltech, he leads an interdisciplinary initiative focused on understanding and measuring intentions in artificial intelligence systems (ai-intentions.org).