DataPipe an interoperability framework to ingest data in InterSystems IRIS in a flexible way.
QuickStart
- Run development container
docker compose up -d
-
Log-in to the system using superuser
/ SYS
.
-
Start an inteoperability production in Interoperability > List > Productions:
DataPipe.Test.Production
- regular objectscript DataPipe demo production
DataPipe.Python.Production
- Python implemented DataPiepe demo production
-
Generate sample data using WebTerminal
do ##class(DataPipe.Test.HL7.Helper).GenerateFilesHL7ADT(100)
DataPipeUI
You can use datapipeUI as a user interface:
- In other directory, clone datapipeUI repository
git clone https://github.com/intersystems-ib/iris-datapipeUI
- Run UI container:
cd iris-datapipeUI
docker compose up -d
- Check the generated sample data using the UI at http://localhost:8080
Ingest data using DataPipe
Model
You need to define a model for the data you want to ingest.
A model a class that extends from DataPipe.Model.cls where you must implement some methods.
In your model you will implement:
- How to serialize / deserialize your data (e.g. using JSON or XML).
- How to Normalize and Validate your data.
- And finally, what operation you want to run with your data after it is normalized and validated.
You can find a full example in DataPipe.Test.HL7.Models.A08.cls
Interoperability components
After defining your model, you can develop your interoperability production using DataPipe components.
You need to implement an Ingestion process which must provide:
Input > InboxAttributes
transformation: here you extract the attributes that describe your input data. You will be able to search data using these attributes.
Input > Model
transformation: transform your incoming data into the DataPipe model you have previously defined.
Rest of the components are provided by DataPipe and they will call your model methods.
You can have a look at a full example in DataPipe.Test.Production.cls
Installation
- Install IPM package manager if you don’t have already done it.
- Create a new namespace (e.g.
DPIPE
)
- Switch to the namespace you want to install DataPipe.
- Install DataPipe using ipm:
DPIPE> zpm
zpm:DPIPE> install iris-datapipe
Want to contribute to this project? See CONTRIB.md