Interop Ops Agent

An IRIS Interoperability workflow where an AI Hub agent makes the decision: what to do with a batch of lab results after the lab quietly changed its HL7 feed.
The AI proposes, the production checks, a human decides.

Online demo (no install) · Ideas Portal: DPI-I-854: AI Log Analytics & Predictive Anomaly Detection for Interoperability (Community Opportunity)

The problem

A hospital lab upgrades its system. Now sodium arrives as SOD instead of NA, or creatinine comes in mg/dL instead of umol/L.

Every message is still valid HL7, so nothing alerts. The interface engine passes the results on, and they land in the wrong place in the chart, or with the wrong number. Often nobody notices until a clinician asks where the results went.

Rules can catch the changes you expect. The hard part is the change nobody wrote a rule for. That is where an AI agent helps, if you do not let it change the feed on its own.

What Interop Ops Agent does

It runs inside an IRIS Interoperability production and watches an HL7 v2 ORU^R01 feed, batch by batch:

1. Rules first. Known patterns (a unit change, a flood of duplicates, a clean batch) are handled by rules, with no AI call.
2. The AI Hub agent takes the rest. It reads the batch with 7 read-only tools and picks one fix from a short list: map a code, convert a unit, quarantine, pass, or escalate to a human.
3. The production checks the fix. The new code must have the right unit and values close to its history, and a replay of 200 old messages must not change.
4. A human decides. The fix waits in the Workflow Inbox. The review card shows what was checked and what was not.
5. The fix is applied. It goes into the lookup table and the batch is reprocessed. If anything fails, nothing is applied and the case is escalated.
6. The next batch needs no AI. The lookup now knows the code, so the same change passes with 0 LLM calls.

Every agent tool call shows up in Visual Trace, and every step is saved as a Decision row.

How this meets the contest task

flowchart LR
    F["HL7 file
cases/in/*.hl7"] --> P["IOA.Process.Message
BPL: DTL + rules"]
    P -->|"resolved"| G["IOA.Gate.Replay
batch + 200 historical messages"]
    P -->|"unresolved"| A["IOA.Agent
%AI.Agent"]
    A <-->|"tool calls"| T["IOA.Agent.Tools
%AI.Tool"]
    A --> G
    G -->|"patch"| W["IOA.Workflow
Workflow Inbox"]
    W -->|"approved"| L["IOA.Lookup.Apply
lookup table + reprocess,
rollback if invalid"]
    G -->|"pass / quarantine / escalate"| D[("Decision")]
    W -->|"rejected"| D
    L --> D
    D --> O["IOA.Deliver
cases/out/ + cases/held/"]

How the project uses AI Hub

The agent must answer with JSON from the fixed list of actions. Message content is treated as data, never as an instruction. Before the agent starts, the production gives it extra evidence: an HL7 check in .NET PEX (NHapi), a UCUM unit check in Embedded Python, and the 3 most similar past incidents from Vector Search.

What we found in AI Hub EAP

• Chat() hands only a provider token to the Rust agent loop, so a %AI.Provider subclass in ObjectScript is never called. REPLAY therefore uses ToolMgr.ExecuteTool directly.
• MaxIterationsReached is returned for max iterations, timeout and loop detection. The code tells them apart by the error text.
• In the audit policy, call.arguments is a JSON string, not a %DynamicObject.

Why the AI does not get the last word

The model makes mistakes. The recordings in replay/ show them, and the production is built so that a mistake does not reach the chart:

• C08, prompt injection. A comment in the message tells the AI to map XQ5 to a glucose code. The model can follow it and propose this fix. The check sees that XQ5 reports in U/L, not in glucose units, and rejects the fix.
• C02. The model can answer “all fine” while an unknown code is in all 30 messages. The check rejects “pass” if anything in the batch is still unexplained.
• C18, whole-blood sodium. The unit and values look exactly like normal sodium. Only the lab’s text says “whole blood”. The agent mapped it to the serum code, and no statistical check can see the difference. So the review card shows a specimen warning, and the human rejects the fix:

To be honest about the numbers: on our 12 demo cases a rules-only baseline also scores 12/12, and on the whole-blood test the agent was no better than the rules. This is why the design does not trust the agent. The checks and the human do. We tested it with blind cases, a pre-registered specimen test and LIVE runs.

Quick start (REPLAY, no API key)

The fastest way is the online demo. It calls a real model (openai/gpt-6-luna, $0.50 per day), so its results can differ from the recorded ones. Try this in 2 minutes:

1. Click Run on C01. The lab now sends sodium as SOD, and the rules cannot explain it, so the agent takes the case.
2. Open the agent’s evidence and the Visual Trace link. You see each tool call and the proposed fix SOD → 2951-2.
3. Click Approve. The fix goes into the lookup table and the batch is reprocessed.
4. Click Run on C01 again. Now the rules decide it with 0 LLM calls.

The demo is shared by all visitors. If someone already approved C01, step 1 already shows the rules result.

For a local run you need Docker with Compose v2 (about 4 GB free RAM), bash, python3, free ports 51272, 52273, 52280, and the IRIS for Health AI Hub Community image, build 2027.1.0AI.109.0, from the EAP portal.

# Put the archive next to the repository (../irishealth_arm64-community-2027.1.0AI.109.0-docker.tar.gz on ARM64,
# ../irishealth-community-2027.1.0AI.109.0-docker.tar.gz on x64), or set IRIS_IMAGE_TAR=/path/to/archive.tar.gz
cd interop-ops-agent
./run.sh                          # loads the image, builds, starts the production; last line: FINAL STATUS: OK
./scripts/demo.sh --replay        # sends the 12 cases, prints decisions, Visual Trace links and "correct 12/12"
./scripts/check.sh                # health checks, including the gate self-test; last line: CHECK: OK
open http://localhost:52273/demo/ # demo page

On Apple Silicon run.sh takes 55 s and demo.sh --replay takes 5 s. REPLAY is the default: the agent’s tool calls come from the recordings in replay/ and go through the same production. If a tool result or the prompt differs from the recording, the case is marked DRIFT and demo.sh exits with code 1.

The demo page /demo/ shows the cases, the agent’s evidence, traces, the Workflow queue and the knowledge base, with a Run button per case and Approve / Reject. To see the whole-blood sodium scene without an LLM:

CASES_SRC=cases/pair EXPECTED_MAP_FILE=cases/pair/expected.json IOA_AUTOAPPROVE=0 ./scripts/demo.sh --manual --case C18 --replay

[!WARNING]
The ports are published on 127.0.0.1 only, because the image has the default _SYSTEM / SYS login. IRIS_BIND=0.0.0.0 publishes them on all interfaces. Change the password before you do that.

[!CAUTION]
The demo page has no login, so any visitor can approve. It is installed only with IOA_DEMO=1 (set in docker-compose.yml). Do not set it on a server with real data.

Live mode

To call a real model, set OPENAI_API_KEY (or LLM_PROFILE=openrouter with OPENROUTER_API_KEY) and IOA_LIVE=1 in .env, then run ./run.sh and ./scripts/demo.sh.

[!IMPORTANT]
LIVE costs money. The default models are gpt-5.4-mini and google/gemini-2.5-flash. ./scripts/demo.sh --live makes one LIVE run, --replay forces REPLAY. Every non-REPLAY run is capped at 60 agent calls and IOA_MAX_USD (default 0.50).

A full LIVE run of the 12 cases on gpt-5.4-mini made 16 LLM calls and cost about $0.04 (2026-11-05, build 109). This is an estimate from token counts at $0.75 / $4.50 per 1M tokens, not checked against a bill.

The 12 synthetic cases

Each case is one batch of 30 ORU^R01 messages (C06: 80) in cases/. The expected outcomes are in cases/expected.json. All patients, values and messages are synthetic.

More cases run only in LIVE: 5 blind cases in cases/blind/ and the specimen pair in cases/pair/.

Other IRIS technologies

Interoperability production with BPL (Production.cls), Workflow with role IOA-Reviewers, a custom outbound adapter (ReviewerNotify.cls), lookup table IOA.LabCodeMap, Embedded Python (Py/Ucum.cls), Vector Search (KB/), .NET PEX (pex/), Docker and IPM (https://github.com/AntonYartsev/interop-ops-agent/blob/main/module.xml; zpm "load /path/to/repo", then IOA_DEMO=1 only if you want the demo page).

Limitations

• Synthetic data only. The agent has not been run on real lab feeds. Only HL7 v2.5.1 ORU^R01.
• Tested on IRIS for Health Community AI Hub EAP 2027.1.0AI.109.0 on ARM64 and x64.
• Delivery is two folders (cases/out/, cases/held/), not a real downstream system. Held messages wait for an engineer. There is no release step.
• The checks cannot tell two tests apart if only the lab’s text differs (for example whole blood and serum). For that, the human reviewer is the only safeguard.
• An IPM install has no .NET runtime, so the .NET HL7 check is off and the agent’s evidence differs from the recordings. There demo.sh --replay marks the 6 agent cases DRIFT (correct 6/12), although the decisions match the expected ones. Use ./run.sh (Docker) for the recorded 12/12.
• An approval is bound to the batch content (SHA-256 at ingest). If the same case id arrives again while the task waits, nothing is applied and the case is escalated.

Commands

Local URLs (web port 52273):

[!NOTE]
Known problems of build 109:

• /csp/irisapp pages return 404: CreateNamespace in a CPF merge does not create the web application. run.sh creates it (Setup.EnsureDeepSee()). After a plain docker compose up -d, run ./run.sh again.
• The image has no curl, only wget.
• <METHOD DOES NOT EXIST>Sharding+2^STU at every start is harmless here.

Public demo host: deploy/.

License

MIT, see https://github.com/AntonYartsev/interop-ops-agent/blob/main/LICENSE.