We’ve discussed ICU alarm fatigue. What about AI fatigue?

We’ve discussed ICU alarm fatigue. What about AI fatigue?

Every booth at conferences says AI. Every panel says AI. Every conversation started with AI. Don’t get me wrong, I’m not a hater. Quite the opposite. In fact, I believe that, used appropriately, AI is a tremendous augment.

The fatigue was palpable. Not because people don’t believe in it. Because they’ve heard the same pitch from a hundred companies running the same foundation models on the same publicly available datasets, and they’re starting to wonder what’s actually different.

Here’s what most of the market hasn’t internalized yet. The model is commodity. GPT, Claude, Gemini, open-source — pick one. The architecture for inference is well-understood. The techniques for fine-tuning are published. None of this is a moat.

The differentiation is the data. Specifically: can you secure access to novel data that nobody else has structured? Data that doesn’t exist in a training set because it was never captured, never persisted, never made computable in the first place.

In healthcare, continuous physiological signal is generally discarded. Not archived. Not downsampled. Gone. The richest, highest-frequency clinical data, the kind that captures what’s actually happening to a patient between documented observations, has never been systematically structured at scale.

The companies that will matter in healthcare AI are not the ones with the best models. They’re the ones who solved the data problem that makes the model useful. The ones who built the infrastructure to capture, normalize, and persist signal that the rest of the industry treats as ephemeral.

The model is the easy part.

An ICU nurse gets hundreds of alarms per shift. Most are non-actionable. The industry’s conversation has shifted to layering AI recommendations everywhere, even on top of the noise. Health systems are rightfully cautious. Deploying AI in critical care is serious business. Critical care is literally a life-safety environment.

Here is the question nobody is asking. When these systems reach the bedside, does the recommendation show its work?

There is a difference between handing a clinician an answer and handing a clinician an evidence chain. One asks for trust. The other asks for judgment.

When an AI recommendation shows up opaque, confident, and unexplained, it does not assist reasoning. It replaces it. The clinician stops evaluating the patient and starts accepting or rejecting a black box. Most people accept. Not because they are lazy. Because the system was designed to make acceptance the path of least resistance.

This is the design choice that determines whether clinical AI upskills or deskills the workforce. Not whether the system recommends. Whether the recommendation is auditable.

In our hospital safety and operations platform, every recommendation to adjust a bedside monitor alarm limit must clear five directly observable safety gates before it reaches a nurse. The nurse sees the evidence chain. She evaluates it against her bedside assessment. She decides what to do with the information we surface.

The system makes the case. The clinician makes the call.

Nobody talks about truth gates in clinical AI. They should. The architecture of the recommendation is the variable that separates a tool from a crutch.