PII Discovery for InterSystems IRIS

In-database PII scanning utility that runs inside InterSystems IRIS via Embedded Python. Uses Microsoft Presidio and spaCy NLP models to identify Personally Identifiable Information across database tables and generates a structured CSV report — all without data leaving the IRIS engine.

Overview

• Non-intrusive — reads metadata and samples data via SQL, no schema changes
• Data sovereignty — runs entirely inside the IRIS process using Embedded Python
• Multi-language — supports Portuguese (PT) and English (EN) PII detection
• Sampled scanning — analyzes up to 100 rows per table by default
• Structured output — CSV report: schema_name,table_name,column_name,pii_type

Architecture

Three decoupled components:

Prerequisites

• Git
• Docker

Installation

Clone the repo and build the Docker image:

git clone https://github.com/intersystems-community/iris-embedded-python-template.git
cd iris-embedded-python-template
docker compose build
docker compose up -d

The build installs all Python dependencies including presidio-analyzer, spaCy, and the NLP models (pt_core_news_sm, en_core_web_sm).

Running the Scan

Default (current namespace, default output path)

docker compose exec iris irispython -m irisapp.pii_discovery

Specify namespace and output path

docker compose exec iris irispython -m irisapp.pii_discovery -n USER -o /tmp/report.csv

CLI Reference

Help

docker compose exec iris irispython -m irisapp.pii_discovery --help

Output

The scan produces a CSV file with the following columns:

schema_name,table_name,column_name,pii_type

Example:

schema_name,table_name,column_name,pii_type
SQLUser,Patients,email,EMAIL_ADDRESS
SQLUser,Patients,ssn,US_SSN
SQLUser,Customers,phone,PHONE_NUMBER

Configuration

Excluded Schemas

System schemas are excluded to avoid scanning IRIS internals. The default set supports wildcard patterns using * suffix:

EXCLUDED_SCHEMAS = {"%SYS", "INFORMATION_SCHEMA", "DOCBOOK", "HSLIB", "Ens*"}

Schemas starting with % are always excluded.

Sample Size

Default is 100 rows per table. Override via PIIScanner constructor:

scanner = PIIScanner(identifier=identifier, sample_size=50)

NLP Models

Running Tests

docker compose exec iris irispython -m pytest python/pii_discovery/tests/ -v

Project Structure

python/pii_discovery/
├── __init__.py          Package init — exports PIIIdentifier, PIIScanner, PIIReporter
├── __main__.py          Entry point — CLI with argparse, orchestrates scan and report
├── pii_identifier.py    PIIIdentifier — wraps presidio_analyzer
├── pii_scanner.py       PIIScanner — IRIS metadata queries + sampling
├── pii_reporter.py      PIIReporter — deduplicates findings and writes CSV
└── tests/
    └── test_pii_discovery.py  Unit tests with mocked dependencies

Ports

Credits

• Microsoft Presidio — PII detection engine
• spaCy — NLP models for Portuguese and English
• InterSystems IRIS Community Edition — Database platform with Embedded Python