DataPipe an interoperability framework to ingest data in InterSystems IRIS in a flexible way.

QuickStart

• Run development container docker compose up -d

• Log-in to the system using superuser / SYS.

• Start an inteoperability production in Interoperability > List > Productions:

• DataPipe.Test.Production - regular objectscript DataPipe demo production

• DataPipe.Python.Production - Python implemented DataPiepe demo production

• Generate sample data using WebTerminal objectscript do ##class(DataPipe.Test.HL7.Helper).GenerateFilesHL7ADT(100)

• Checkout interoperability production DataPipe.Test.Production.

DataPipeUI

You can use datapipeUI as a user interface:

1. In other directory, clone datapipeUI repository

git clone https://github.com/intersystems-ib/iris-datapipeUI

1. Run UI container:

cd iris-datapipeUI docker compose up -d

1. Check the generated sample data using the UI at http://localhost:8080

Ingest data using DataPipe

Model

You need to define a model for the data you want to ingest. A model a class that extends from DataPipe.Model.cls where you must implement some methods.

In your model you will implement: * How to serialize / deserialize your data (e.g. using JSON or XML). * How to Normalize and Validate your data. * And finally, what operation you want to run with your data after it is normalized and validated.

You can find a full example in DataPipe.Test.HL7.Models.A08.cls

Interoperability components

After defining your model, you can develop your interoperability production using DataPipe components.

You need to implement an Ingestion process which must provide: * Input > InboxAttributes transformation: here you extract the attributes that describe your input data. You will be able to search data using these attributes. * Input > Model transformation: transform your incoming data into the DataPipe model you have previously defined.

Rest of the components are provided by DataPipe and they will call your model methods.

You can have a look at a full example in DataPipe.Test.Production.cls

Installation

1) Install IPM package manager if you don't have already done it. 2) Create a new namespace (e.g. DPIPE) 3) Switch to the namespace you want to install DataPipe. 4) Install DataPipe using ipm:

DPIPE> zpm zpm:DPIPE> install iris-datapipe

Want to contribute to this project? See CONTRIB.md