LabTriage Agent

Entry for the InterSystems Developer Community contest “AI Agents in Interoperability Workflows” (2026).

Triage of HL7 v2 ORU^R01 lab results on InterSystems IRIS for Health Interoperability.
Deterministic rules decide CRITICAL. An AI agent (%AI.Agent, AI Hub SDK) explains the abnormal
results and decides RELEASE or HOLD_FOR_REVIEW. The most severe decision of the message wins.

All data in this repo is synthetic. The critical limits are fictional. Not for clinical use.

Article on the Developer Community: Rules First, Model Second, Human Last: Triaging HL7 v2 Lab Results with an AI Hub Agent

Community Idea implemented

DPI-I-689: Use InterSystems Interoperability as a Traceability tool for GenAI LLM pipelines

Every LLM call of the production is an Interoperability message: the TriageProcess sends an AgentRequest to the
AgentOperation, the only component that talks to the LLM provider, and gets an AgentResponse back. Each call is
therefore persisted in the message store and shown in Visual Trace, in the same session as the lab result
(TriageRequest), the triage output and, for held messages, the review task. No separate tracing SDK or service is
involved.

Metadata visible in Visual Trace:

Vector Search is not used: the idea suggests it, but it is not required.

See Reproducible example: LLM traceability in Visual Trace.

Comparison with Langfuse

Langfuse is the better choice for teams that need evaluation pipelines, cost dashboards and prompt management across many LLM applications. The LabTriage approach fits a narrower but common case: the LLM is one step inside an interoperability workflow, and the goal is to see that step in the same trace, database and tooling as the rest of the message flow, with no extra service to deploy. The two are not mutually exclusive.

Prerequisites

• Docker and Docker Compose v2.24+ (env_file with required: false)
• x86_64 host: the image is linux/amd64 only (Apple Silicon runs it emulated)
• Access to the InterSystems AI Hub Early Access Program (EAP)

Base image

This app was built and tested on the AI Hub Early Access build 2026.3.0AI.154.0 (IRIS for Health Community,
Build 154U), set in line 1 of the Dockerfile (ARG IMAGE=irishealth-community:2026.3.0AI.154.0).

The EAP portal now also offers 2027.1 (Community build 109). The organizers said the judges will load the build
that matches each app; for this app, that is 2026.3.0AI.154.0.

The contest rules require this base image, and no public registry has it. Load it once:

1. Download irishealth-community-2026.3.0AI.154.0-docker.tar.gz (x64) from the InterSystems AI Hub
   Early Access Program (EAP) portal.
2. Load and retag it:

docker image load -i irishealth-community-2026.3.0AI.154.0-docker.tar.gz
docker tag docker.iscinternal.com/docker-intersystems/intersystems/irishealth-community:2026.3.0AI.154.0 \
  irishealth-community:2026.3.0AI.154.0

Quick start

git clone https://github.com/AzizCotrim/iris-lab-triage-agent.git
cd iris-lab-triage-agent
docker compose up -d --build

• The first build takes about 4 minutes.
• The production LabTriage.Production starts on its own when the container starts (1 to 2 minutes in REPLAY mode).
• docker compose ps shows iris as healthy once the production is running.
• You don’t need a .env file: the default mode is REPLAY (recorded LLM answers, no API key).

LLM provider

Gemini through the AI Hub SDK, model gemini-3.5-flash-lite by default.

To use the real model:

cp .env.example .env
# set GOOGLE_API_KEY= and LABTRIAGE_AGENT_MODE=REAL
docker compose up -d

LABTRIAGE_MODEL (optional) changes the model without a recompile.

Agent modes (LabTriage.Operation.AgentOperation):

Which mode runs: the Mode setting of AgentOperation, if set. Otherwise LABTRIAGE_AGENT_MODE. Otherwise REPLAY.

Note: REAL mode rewrites data/replay/ on every OK answer. Run git restore data/replay after a REAL run.

Verify it works

One sample by hand:

cp data/samples/13-critical-and-abnormal.hl7 data/in/
# a few seconds later:
ls data/out/*/
cat data/out/CRITICAL/SYN-ORU-0013.final.json

All 20 samples (needs python3). The script sets Mode=REPLAY, deletes the files in data/out and data/error,
drops every sample into data/in and checks each result (about 1 minute):

tests/e2e/samples.sh

Human review (needs python3): completes the open review tasks, then sends samples 06 and 02 through the Workflow
review (Release and KeepOnHold) and checks the tasks and the .review.json files (about 30 seconds). It also
deletes the files in data/out and data/error:

tests/e2e/workflow.sh

Unit tests (96 %UnitTest tests):

docker compose exec -T iris iris session IRIS -U IRISAPP <<'EOF'
Set ^UnitTestRoot="/home/irisowner/dev/tests/UnitTest"
Do ##class(%UnitTest.Manager).RunTest("LabTriage","/recursive/nodelete")
Halt
EOF

Management Portal: http://localhost:52773/csp/sys/UtilHome.csp, user SuperUser, password SYS
(local dev credentials).
Production page: http://localhost:52773/csp/healthshare/irisapp/EnsPortal.ProductionConfig.zen?PRODUCTION=LabTriage.Production

Reproducible example: LLM traceability in Visual Trace

Every LLM call of the production is an Interoperability message: the TriageProcess sends an
AgentRequest to the AgentOperation and gets an AgentResponse back. One command shows three
cases in REPLAY mode (no API key, no new recordings):

tests/e2e/trace-example.sh

It drops the 3 messages of data/trace-example/ into data/in, checks each session in
Ens.MessageHeader, and prints the SessionId, the final decision and a Visual Trace link for each
case. It sets the AgentOperation Mode back to its previous value at the end.

Where to click. Open the printed link (user SuperUser, password SYS). In the sequence diagram:

• click the AgentResponse arrow (AgentOperation → TriageProcess) and open the Body tab:
  PromptVersion, InputTokens, OutputTokens, LatencyMs, Provider, Model and Status;
• click the AgentRequest arrow (TriageProcess → AgentOperation): Attempt is the retry number,
  and Payload is the exact sanitized JSON sent to the model (no patient identity);
• click the TriageResponse arrow (TriageProcess → DecisionFileOperation): AgentAttempts,
  AgentStatus, AgentLatencyMs and, per exam, DecidedBy / ReasonCode (FALLBACK / AGENT_FALLBACK
  in case c).

In REPLAY, the metadata comes from the recording of the original REAL call. Cases b and c open a
review task each in the Workflow Inbox (samples.sh and workflow.sh complete them).

Case b (SYN-TRACE-0002): the AgentResponse open in the Body tab, with PromptVersion, InputTokens /
OutputTokens and LatencyMs.

How it works

flowchart LR
  IN["data/in/*.hl7"] --> FI["FileInbound
parse ORU^R01"]
  FI -- "invalid file" --> EFO["ErrorFileOperation"] --> ERR["data/error/"]
  FI -- "TriageRequest" --> TP["TriageProcess
rules"]
  TP -- "abnormal exams
(sanitized)" --> AO["AgentOperation
%AI.Agent"]
  AO -- "AgentResponse" --> TP
  TP -- "CRITICAL_ALERT, then FINAL" --> DFO["DecisionFileOperation"] --> OUT["data/out/"]
  TP -- "HOLD_FOR_REVIEW:
TaskRequest (async)" --> LR["LabReviewer
Workflow Operation"]
  LR <--> INBOX["Workflow Inbox
(reviewer)"]
  LR -- "TaskResponse
(after the action)" --> TP
  TP -- "ReviewResult" --> DFO

For each exam (OBX), in this order:

1. FileInbound marks the exams it cannot evaluate (first problem wins). They go to review.
2. Critical rule (LabTriage.Rules.CriticalLimits): fictional limits from LabTriage.ExamCatalog, or a critical
   laboratory flag (HH, LL, AA). If any exam is critical, the alert file is written before the agent is called.
3. Reference range (OBX-7: a-b, <x, >x) plus the flag: normal is released, unreadable goes to review,
   abnormal goes to the agent.
4. Agent: one call per message for all its abnormal exams. The payload has only code, name, value, unit,
   range, flag and status: no patient identity, order or free text (LabTriage.Util.Sanitizer).
   LabTriage.Util.AgentContract validates the answer. A bad answer, timeout or provider error gets one retry.
   If there is still no valid answer, the exams go to review (fallback).

The message decision is the most severe one: CRITICAL > HOLD_FOR_REVIEW > RELEASE.

Supported exams: hemoglobin (718-7), glucose (2345-7), potassium (2823-3), creatinine (2160-0), sodium (2951-2).
You can edit the flag categories in the Lookup Table LabTriage.FlagCategory (defaults in config/FlagCategory.xml).

Main reason codes:

Data folders (data/):

Output

data/out/CRITICAL/SYN-ORU-0013.final.json (sample 13, REPLAY mode):

{
  "messageControlId": "SYN-ORU-0013",
  "outputType": "FINAL",
  "orderId": "ORD-1013",
  "patientId": "SYN-000113",
  "decision": "CRITICAL",
  "released": [
    {"code": "2160-0", "name": "Creatinine", "decidedBy": "RULE", "reasonCode": "NORMAL",
     "reason": "Within reference range."}
  ],
  "review": [
    {"code": "2345-7", "name": "Glucose", "decidedBy": "AGENT", "reasonCode": "AGENT_FLAGGED",
     "reason": "Glucose is clearly outside its reference range, requiring review of exam 2345-7."}
  ],
  "critical": [
    {"code": "2823-3", "name": "Potassium", "decidedBy": "RULE", "reasonCode": "CRITICAL_HIGH",
     "reason": "Potassium 6.8 mmol/L is above the critical high limit of 6.2 mmol/L (fictional limit)."}
  ],
  "agent": {"called": true, "attempts": 1, "status": "OK", "latencyMs": 798,
            "provider": "replay", "model": "gemini-3.5-flash-lite"},
  "emittedAt": "2026-11-03T20:20:54.074Z"
}

data/out/ALERTS/SYN-ORU-0013.critical.json has the same shape, with "outputType": "CRITICAL_ALERT" and only the
critical list. All times are UTC.

data/error/10-malformed-no-obx.hl7.error.json:

{
  "sourceFileName": "10-malformed-no-obx.hl7",
  "reason": "The message has no OBX segment.",
  "messageControlId": "SYN-ORU-0010",
  "patientId": "SYN-000110",
  "receivedAt": "2026-10-31T11:34:09Z"
}

Human review (Workflow)

A message whose final decision is HOLD_FOR_REVIEW also becomes a task in the Workflow Inbox, for the
LabReviewer role. This includes the agent fallback (no valid answer from the agent). CRITICAL messages don’t get a
task: the alert file is their escalation. RELEASE messages need no human.

• Inbox: http://localhost:52773/csp/healthshare/irisapp/_DeepSee.UserPortal.Workflow.zen, user SuperUser,
  password SYS. The Inbox shows the tasks of the logged-in user only. iris.script creates the Workflow users
  SuperUser and _SYSTEM at build time and adds both to LabReviewer, so each task reaches both of them.
• The task subject is Lab results held for review: MSH-10 <MSH-10>, order <OBR-2>. The message lists the exams held
  for review, with their reasonCode and reason.

To review a task:

1. Open the task in the Inbox.
2. Click Accept. The task is now yours (the other user can no longer act on it).
3. Click Release or KeepOnHold.

The action goes to data/out/REVIEWED/<MSH-10>.review.json (sample 06, Release):

{
  "messageControlId": "SYN-ORU-0006",
  "outputType": "REVIEW",
  "orderId": "ORD-1006",
  "patientId": "SYN-000106",
  "action": "Release",
  "reviewer": "SuperUser",
  "role": "LabReviewer",
  "taskCreatedAt": "2026-11-01T13:11:55.082Z",
  "taskCompletedAt": "2026-11-01T13:11:58.836Z",
  "emittedAt": "2026-11-01T13:11:58.837Z"
}

After tests/e2e/samples.sh, the Inbox has one task per HOLD_FOR_REVIEW sample (10 tasks), ready to review.
The next run of tests/e2e/samples.sh or tests/e2e/workflow.sh first completes with Release the tasks still open
(tests/e2e/drain-review-tasks.sh), so their .review.json files are written and then deleted with the rest of
data/out.

The file name rules are the same as for .final.json: safe characters only, and a timestamp suffix for a repeated MSH-10.

The review never blocks or changes the triage. The .final.json is written right away, and the review is one more
event later, in its own file. The .final.json stays as it was.

The call is asynchronous: the Workflow Operation (EnsLib.Workflow.Operation) defers its response (DeferResponse)
until a reviewer acts, however long that takes. The TriageProcess sends the task with SendRequestAsync and writes the
ReviewResult in OnResponse. If the task cannot be created (OnError), the process only logs a warning.

Known limitations

• Reference ranges with a negative lower bound (e.g. -1-5) are not parsed (UNPARSEABLE_RANGE).
• The agent contract validates the structure of the answer, not the medical content of reason.
• If an exception happens before the critical rule evaluates an exam, that exam goes to review, not CRITICAL.
• MSH-10, PID-3 or OBR-2 longer than 255 characters are rejected.
• The same rejected file name twice overwrites the previous .error.json.
• REPLAY only has recordings for the abnormal exams of the samples; any other abnormal input falls back to HOLD_FOR_REVIEW.
• data/in, out, error, archive and replay are chmod 0777 by the init-perms service (local demo, synthetic data).
• Lookup Table edits made in the Management Portal are lost when the container is recreated: the table is
  reloaded from config/FlagCategory.xml at build time.
• The IRIS instance status shows “alert” because of startup errors in the EAP build
  (<METHOD DOES NOT EXIST>Sharding+2^STU, “System appears to have failed over from node buildkitsandbox”).
  The compose healthcheck reports the production state instead.
• Review tasks have no timeout or expiration: a task stays open until a reviewer acts on it.
• Each open review task keeps one TriageProcess instance persisted and waiting for the answer.
• The Inbox action buttons are disabled while the production is stopped.
• The Priority column of the Inbox is empty for the review tasks (cause not investigated).
• linux/amd64 only.

Tests

The e2e scripts need the iris service up. They change the AgentOperation Mode on the running production and set it
back to empty at the end (trace-example.sh sets back the value it found). clean-clone-check.sh stops this repo’s stack first and does not restart it.

Project structure

├── Dockerfile, docker-compose.yml, iris.script, merge.cpf
├── .env.example
├── config/FlagCategory.xml        flag -> NORMAL / ABNORMAL / CRITICAL defaults
├── data/                          in, out, error, archive, replay, samples, trace-example
├── docs/dev-journal.md            what was built and observed, day by day
├── docs/images/                   README screenshots
├── src/LabTriage/
│   ├── Production.cls
│   ├── Service/FileInbound.cls
│   ├── Process/TriageProcess.cls
│   ├── Operation/                 AgentOperation, DecisionFileOperation, ErrorFileOperation
│   ├── Agent/TriageAgent.cls      %AI.Agent subclass: prompt, provider, model
│   ├── Rules/CriticalLimits.cls
│   ├── Util/                      ReferenceRange, AgentContract, Sanitizer, FlagTable
│   ├── ExamCatalog.cls
│   └── Msg/                       message classes
└── tests/                         UnitTest/ and e2e/

AI Hub bugs reported

• #9: the OpenAI-compatible
  providers (openai, gemini) drop finish_reason, so a reply cut at the token limit looks like a complete answer.
• #10: provider errors that AI Hub
  recognises (model not found, billing, timeout) are thrown as <%AICore>ProviderError instead of their dedicated codes.

Both were found while building LabTriage Agent.

License

MIT, see https://github.com/AzizCotrim/iris-lab-triage-agent/blob/main/LICENSE.