The boundary, made concrete
The design philosophy is uncompromising: the AI assists, humans decide. Every AI component is bounded to a narrow mandate, every output is explainable and logged, and the entire layer can be disabled with the platform continuing to operate as a deterministic system.
That last property is the one that matters under scrutiny. A platform whose AI cannot be switched off without the platform failing has not implemented human oversight; it has implemented a dependency.
- Orchestration
- A backend orchestrator decides when a model runs, assembles its grounded context, and routes its output to a human queue — never directly to a clinical or dispatch decision.
- Grounded reasoning
- Models reason over retrieved, approved reference material rather than free recall, which keeps outputs grounded and auditable.
- Guardrails
- Structured outputs, validation of every response, deterministic fallback, and a global kill-switch.
- Auditability
- Each AI action is written to an immutable log with its inputs, outputs and rationale, and is reconstructible by incident.
- Human-in-the-loop enforcement
- Enforced in the orchestrator, not left to good intentions or to training. A configuration that removes human review is not available to deploying organisations.
What the AI is never permitted to do
- No aircraft is launched by the platform. Dispatch is recommended by the system and committed by a named authoriser, with the reason recorded.
- No diagnosis is made. Triage support suggests an order; a clinician confirms it.
- No aid request is declined by a model. Verification decisions that refuse a request are made by a person, and carry an appeal route.
- No consent is inferred. Emergency access paths exist and are recorded; they are never granted by a model's judgement of clinical necessity.
- No resource is committed autonomously. Predictive output shapes standby posture and pre-positioning; commitment is a human act.
- No model output is presented without its uncertainty. A prediction offered without a confidence range invites false precision.
Oversight function
SAHI recommends dedicated AI units or nodal cells within health departments and institutions, to lead AI strategy, use-case prioritisation, tool assessment, deployment oversight and lifecycle management.
The programme maintains an AI oversight function with a named accountable owner, an agent audit log reviewable by compliance, per-component enable and disable controls, and a change-audit note on every configuration change. Deploying organisations are supported in establishing the equivalent function on their own side, because oversight that exists only at the vendor is not oversight.