Initial Release
Parse and chart InterSystems Caché pButtons and InterSystems IRIS SystemPerformance files.
For combining metrics from mgstat, iostat and vmstat see Pretty Performance below.
What do all these metrics mean? See my posts on the Intersystems Community
This is a replacement for yape. There will be no more development on yape.
I will add functionality to yaspe as I need it. I use these tools pretty much daily, so there are frequent updates; which is not the same as support ;)
However, if you find bugs, or are looking for enhancements let me know!
NOTE:
yaspecurrently only supports:
- IRIS/Caché (mgstat)
- Linux (vmstat, iostat)
- Windows (Perfmon)
- AIX (vmstat, iostat, sar -d)
Build the yaspe image on your own system — it works the same on Apple silicon and Intel:
git clone https://github.com/murrayo/yaspe.git, or download and unzip the source)cd to the folder with the source files (keep it separate from the folder holding your SystemPerformance files)yaspe container image: docker build --no-cache -t yaspe .No Docker Hub account or docker login is needed — the build is entirely local.
There are also instructions for running without docker
See the help text:
docker run -v "$(pwd)":/data --rm --name yaspe yaspe ./yaspe.py -h
Linux hosts: the container runs as root, so output files land owned by root.
Add--user "$(id -u):$(id -g)"to anydocker runcommand to have outputs
owned by you. (Not needed on Docker Desktop for macOS or Windows.)
usage: yaspe [-h] [-v] [-i "/path/file.html"] [-x] [-n] [-a] [-o "output file prefix"] [-e "/path/filename_SystemPerformance.sqlite"] [-c] [-p] [-P] [--dots] [-s] [-m] [-D] [-d DISK_LIST [DISK_LIST ...]] [--all-disks] [--iostat_no_subfolders] [-l "string to split on"] [--peak_chart] [--no_peak_chart] [-C "/path/to/directory"] [-B] [--smooth-minutes N] [--day-overlay] [--bh-charts] [--long-period-smooth N] [--context "context string"] [--llm-context] [--resample INTERVAL]Performance file review.
options: -h, --help show this help message and exit -v, --version show program's version number and exit -i "/path/file.html", --input_file "/path/file.html" Input HTML or .mgst filename with full path. -x, --iostat Also chart iostat data (this can take a long time). -n, --nfsiostat Also chart nfsiostat data. -a, --append Do not overwrite database, append to existing database. -o "output file prefix", --output_prefix "output file prefix" Output filename prefix, defaults to HTML file name, blank (-o '') is legal. -e "/path/filename_SystemPerformance.sqlite", --existing_database "/path/filename_SystemPerformance.sqlite" Chart existing database, full path and filename to existing database. -c, --csv Create CSV files of each HTML files metrics, append if csv file exists. -p, --png Create PNG charts of metrics. No HTML. HTML is the default if PNG not selected. -P, --PNG Create PNG and HTML charts of metrics. Charts are written into png/ and html/ subdirectories within each metric folder. --dots Create PNG charts as dot charts instead of line charts (default is lines). -s, --system Output system overview. -m, --mgstat_file This is an mgstat file log file (with extension .mgst). -D, --DDMMYYYY Date format for csv files is DDMMYYYY -d DISK_LIST [DISK_LIST ...], --disk_list DISK_LIST [DISK_LIST ...] List of disks, if not entered all are processed. No commas or quotes, e.g. -d dm-0 dm-1 On Windows,
-dfilters perfmon disk counters by drive letter (e.g.-d F: J:): only matching PhysicalDisk/LogicalDisk columns are stored, plus_Totaland all non-disk counters. --all-disks Store every iostat device (Linux) or perfmon disk counter (Windows) in the SQLite database. By default, when a CPF file is found in the SystemPerformance HTML, only IRIS-related disks (database, primary/alternate journal, WIJ devices — iostat devices on Linux, drive letters on Windows) are stored, which makes extraction much faster and databases much smaller. Use --all-disks (or an explicit -d list) when you need to investigate non-IRIS devices — re-running extraction is cheap. --iostat_no_subfolders Save all iostat charts flat (no per-device subfolders). Default is to use subfolders. -l "string to split on", --large_file_split_on_string "string to split on" Split large input file on first occurrence of this string. Blank -l "" defaults to "div id=iostat" --peak_chart Create additional peak 60-minute charts for metrics with min_max enabled when data is 8-25 hours. Default is True. --no_peak_chart Disable peak 60-minute charts. -C "/path/to/directory", --compare-dir "/path/to/directory" Compare all HTML files in a directory: produce vmstat and mgstat overlay charts. -B, --combined Also create a combined vmstat+mgstat overlay HTML chart alongside all other charts. Also runs automatically in default HTML and -P modes; combined_overlay.html is written to {prefix}_metrics/. --smooth-minutes N Rolling average window in minutes for --combined chart (default: 5, 0 = raw). --day-overlay Create day-overlay charts for all metrics when data spans more than 25 hours. Total CPU, Glorefs, and PhyRds always get day-overlay charts regardless of this flag. --bh-charts Create per-day business-hours peak charts for multi- day data (slow; off by default). --long-period-smooth N Rolling average window in minutes for multi-day charts (default: 5). --context "context string" Optional context note included in the LLM context bundle (e.g. "users reported slowness Tuesday"). --llm-context Export an anonymized markdown context bundle plus analysis prompt for LLM-based performance review (implies -s). --resample INTERVAL Resample interval for timeseries in the LLM context bundle. Default: auto — 5min for up to 2 days of data, 15min for 3-4, 30min for 5+. Examples: 5min, 10min, 30min.
Be safe, "quote the path".
For example, change to the folder with a SystemPerformance html file and run the command:
docker run -v "$(pwd)":/data --rm --name yaspe yaspe ./yaspe.py -i /data/mysystems_systemperformance_24hour_1sec.html
If you want simple png files rather than html: smaller and quicker to look through: Use the -p option.
docker run -v "$(pwd)":/data --rm --name yaspe yaspe ./yaspe.py -i /data/mysystems_systemperformance_24hour_1sec.html -p
When a CPF file is found in the SystemPerformance HTML, yaspe automatically stores only the IRIS-related disks — database, primary/alternate journal, and WIJ devices. On Linux these are iostat devices (e.g. dm-7); on Windows they are perfmon drive letters (e.g. F:). This makes extraction much faster and databases much smaller (a 24-hour file with 72 disks drops from ~55 s / 154 MB to ~7 s / 16 MB).
You will see which disks were selected in the output:
Auto disk list from CPF (extraction): ['dm-18', 'dm-17', 'dm-7', 'dm-8']
To override the automatic selection:
-d — choose the disks yourself. A list of devices without the /dev/ on Linux, or drive letters on Windows:docker run -v "$(pwd)":/data --rm --name yaspe yaspe ./yaspe.py -i /data/mysystems_systemperformance_24hour_1sec.html -p -x -d dm-0 dm-1
--all-disks — store every disk (the pre-v0.11 behaviour), for example when investigating a non-IRIS device. Re-running extraction is cheap.Iostat charts are saved into per-device subfolders by default, creating {prefix}_metrics/iostat/dm-0/, {prefix}_metrics/iostat/dm-1/, etc. To disable this and place all disk charts flat in a single iostat/ folder, add --iostat_no_subfolders:
docker run -v "$(pwd)":/data --rm --name yaspe yaspe ./yaspe.py -i /data/mysystems_systemperformance_24hour_1sec.html -p -x --iostat_no_subfolders
Or put the path to the folder with the html file in the docker volume parameter and put the html file name after -i /data/
docker run -v "/path/to/folder/with html file":/data --rm --name yaspe yaspe ./yaspe.py -i /data/mysystems_systemperformance_24hour_1sec.html
To run yaspe over multiple input files, for example a few days or a week, use the following steps:
-a (append) option to put all the metrics in the database (also -x if you want iostat)-e (existing database option) to chart the appended database (also -x if you want iostat)Note: This works by appending data to a database that contains extracted SystemPerformance data.
Since iostat is filtered to IRIS disks by default (see Disk filtering), including -x over a full week is fast — a week of 24-hour, 5-second-interval files extracts in well under a minute.
If the resulting charts have too many data points to be comfortable in the browser, deep dive on a day or two using the method above.
When the combined database spans more than 25 hours, yaspe automatically generates additional supplementary charts for key metrics (see Long-period charts below): 5-minute and 30-minute smoothed views, a daily 99th percentile bar chart, an hourly heatmap, and per-day business hours peak charts. Day-overlay charts are always produced for Total CPU, Glorefs, and PhyRds; use --day-overlay to produce them for all metrics.
By default, output folders and files are prefixed with the html file name.
To keep all the metric data in a single database use the -o argument to override the output file prefix.
Example of running over multiple days;
for i in *.html; do docker run -v "$(pwd)":/data --rm --name yaspe yaspe ./yaspe.py -i /data/"${i}" -a -o "three_days"; done
The resulting database file will use the prefix, in this example; three_days_SystemPerformance.sqlite
To create charts for the accumulated days use the -e argument.
docker run -v "$(pwd)":/data --rm --name yaspe yaspe ./yaspe.py -e /data/three_days_SystemPerformance.sqlite
./prefix_metrics. Each chart has a main zoom panel and an overview panel — drag the overview to zoom, double-click to reset. A combined_overlay.html (vmstat CPU + mgstat IO) is also written automatically to {prefix}_metrics/.-p): static PNG charts. Use for quick review or when sharing files that will not be opened in a browser.-P): produce both formats. PNG files go into png/ and HTML files go into html/ subdirectories within each metric folder (e.g. vmstat/png/, vmstat/html/). A combined_overlay.html is also written automatically to {prefix}_metrics/.-x). Since disks are filtered to IRIS devices by default this is quick; with --all-disks and a large disk list it can take a long time.-o your_choice or -o '' for no prefix.-c argument. If you use -c with -o csv files (for example for multiple days) will append.When data spans more than 25 hours (e.g. a week of appended SystemPerformance files), yaspe generates a set of supplementary PNG charts in addition to the main chart. These are produced for metrics that have min/max enabled (key metrics such as Total CPU, Glorefs, WIJwri, etc.):
| Chart | Filename suffix | Description |
|---|---|---|
| 5-minute average | z_{metric}_5min_avg.png |
Raw samples shown faintly behind a 5-minute rolling mean. Legend shows original sample interval and smoothing window (e.g. Total CPU (5m samples, 5 min avg)). |
| 30-minute average | z_{metric}.png |
Same layout smoothed to a 30-minute rolling mean — the main long-period PNG. |
| Daily 99th pct bar chart | z_{metric}_daily_summary.png |
One bar per calendar day, coloured red for the busiest day. |
| Hourly heatmap | z_{metric}_heatmap.png |
Hour-of-day × date grid, colour-coded by 99th percentile value. Shows consistent peak hours across days. |
| Day-overlay PNG | z_{metric}_day_overlay.png |
All days overlaid on a shared 00:00–24:00 x-axis, one colour per day. Always produced for Total CPU, Glorefs, and PhyRds; produced for all other metrics only when --day-overlay is passed. |
| Day-overlay HTML | {metric}_day_overlay.html |
Interactive version of the day-overlay chart (produced with -P or HTML-only mode). Same conditions as the PNG above. |
| Per-day business hours peak | z_{metric}_bh_peak_{date}.png |
Business hours (08:00–18:00) peak 60-minute window for each day. Produced only when --bh-charts is passed (off by default — can be slow). |
For single-day or short multi-day runs (8–25 hours), yaspe produces:
z_{metric}_peak.png): zooms to the busiest 60-minute window of the day for key metrics.z_{metric}_bh_peak.png): same as above but restricted to business hours (08:00–18:00).Example output

To update, remove the old image, download the latest source, and rebuild:
docker rmi yaspe
docker build --no-cache -t yaspe .
yaspe includes a system overview and basic config check (-s)
This check is designed to save you hunting through your SystemPerformance file looking for system details.
[prefix]_overview_all.csv[prefix]_overview.txtAn example of overview.txt follows, the contents will be different on your system:
System Summary for your site name
Hostname : YOURHOST
Instance : SHADOW
Operating system : Linux
Platform : N/A
CPUs : 24
Processor model : Intel(R) Xeon(R) Gold 6248 CPU @ 2.50GHz
Memory : 126 GB
Shared memory : globals 71680 MB + routines 1023 MB + gmheap 1000 MB = 73,703 MB
Version : Cache for UNIX (Red Hat Enterprise Linux for x86-64) 2018.1.4 (Build 505_1U) Thu May 28 2020 10:11:16 EDT
Date collected : Profile run "24hours" started at 16:15:00 on Nov 22 2021.
Warnings:
- ** Insecure Private Webserver Enabled! **
- Journal freeze on error is not enabled. If journal IO errors occur database activity that occurs during this period cannot be restored.
- swappiness is 10. For databases 5 is recommended to adjust how aggressive the Linux kernel swaps memory pages to disk.
- Hugepages not set. For performance, memory efficiency and to protect the shared memory from paging out, use huge page memory space. It is not advisable to specify HugePages much higher than the shared memory amount because the unused memory are not be available to other components.
- dirty_background_ratio is 10. InterSystems recommends setting this parameter to 5. This setting is the maximum percentage of active memory that can be filled with dirty pages before pdflush begins to write them.
- dirty_ratio is 30. InterSystems recommends setting this parameter to 10. This setting is the maximum percentage of total memory that can be filled with dirty pages before processes are forced to write dirty buffers themselves during their time slice instead of being allowed to do more writes. These changes force the Linux pdflush daemon to write out dirty pages more often rather than queue large amounts of updates that can potentially flood the storage with a large burst of updates
Recommendations:
- Review and fix warnings above
- Set HugePages, see IRIS documentation: https://docs.intersystems.com/irislatest/csp/docbook/Doc.View.cls?KEY=GCI_prepare_install#GCI_memory_big_linux
- Total memory is 128,755 MB, 75% of total memory is 96,566 MB.
- Shared memory (globals+routines+gmheap) is 73,703 MB. (57% of total memory).
- Number of HugePages for 2048 KB page size for (73,703 MB + 5% buffer = 77,388 MB) is 38694
All instances on this host:
SHADOW 2018.1.4.505.1.a 56772 /cachesys
--llm-context exports two files alongside the SQLite database:
performance_context_{start}_{end}.md — an anonymized data bundle:llm_analysis_prompt.md — the companion prompt: methodology,Unlike every other yaspe output, these two filenames never carry the
-o output prefix — -o normally defaults to the input HTML filename, which
typically embeds the hostname and instance, and that name would otherwise
travel with the file when you upload it to a public LLM. Only the dates in
the bundle filename come from the data.
Attach both files to your LLM chat of choice. Use --context "note" to
embed a free-text note (it is redacted like everything else) and --resample
to override the timeseries interval (default: auto — 5min for up to 2 days,
15min for 3–4, 30min for 5+, so multi-day bundles stay LLM-sized).
./yaspe.py -e yaspe_SystemPerformance.sqlite --llm-context -o yaspe
Anonymization is best-effort — eyeball the bundle before sharing it
externally.
docker run -v "$(pwd)":/data --rm --name yaspe yaspe ./yaspe.py -i /data/SystemPerfomanceFileName.html -a -s -x -o yaspe
docker run -v "$(pwd)":/data --rm --name yaspe yaspe ./yaspe.py -e /data/yaspe_SystemPerformance.sqlite -p
docker run -v "$(pwd)":/data --rm --name yaspe yaspe ./yaspe.py -e /data/yaspe_SystemPerformance.sqlite -o html
Next steps:
yaspe is written in Python.
If you wish to run locally in the operating system, I suggest you create a separate Python virtual environment for yaspe.
There are many references for creating Python environments on the web. You can start with the official documentation:
yaspe is tested in Python 3.12. Specifically my test system is:
python --version
Python 3.12.13
Once you have set up your Python environment:
cd to folder with source filesSee requirements.txt for the Python packages used. You will need to:
pip3 install -r requirements.txt
Once installed you can cd to the folder with your pButtons or SystemPerformance file(s) and run the following sequence:
for i in `ls *.html`; do /path/to/yaspe/souce/you/downloaded/yaspe.py -i $i -a -s -x -o yaspe;done
/path/to/yaspe/souce/you/downloaded/yaspe.py -e yaspe_SystemPerformance.sqlite -p
/path/to/yaspe/souce/you/downloaded/yaspe.py -e yaspe_SystemPerformance.sqlite -o html
Below is the example custom chart, Glorefs (mgstat) and Total CPU utilisation (vmstat).

Below is one of the default images, which includes a zoom to specified time (or defaults to 13:00-14:00).

Another custom image included in the sample charts.yml file. This chart combines read and write latency for IRIS disks. (Awful WIJ write performance!)

Here is the same chart with all metrics on the left axis only (not as clear, but depends on the metrics).
_1000_to_1100_-_Sample_site_name_Monday_17_Jan_2022.png)
Maybe you care about IO throughput for a cloud disk…

pretty_performance uses the sqlite database created by yaspe to make charts that can combine metrics for vmstat, iostat and mgstat.
For example, this is handy if you need to output charts for performance reports.
There is also an option to output merged vmstat, iostat and mgstat as a csv file for you to work with in Excel or other ways.
Formatting and chart creation is driven from two yml files, I have included samples;
docker run -v "$(pwd)":/data --rm --name yaspe yaspe ./pretty_performance.py -h usage: pretty_performance.py [-h] -f DB_FILENAME [-s ZOOM_START] [-e ZOOM_END] -p PARAMATER_FILE [-i] [-m] [-x] [-c CHART_FILE] [-o OUTPUT_DIR] [-l]Create charts from Linux pButtons and SystemPerformance data already stored in sqlite3 file
optional arguments: -h, --help show this help message and exit -f DB_FILENAME, --db_filename DB_FILENAME db path and file name -s ZOOM_START, --zoom_start ZOOM_START Start time for zoom -e ZOOM_END, --zoom_end ZOOM_END Stop time for zoom -p PARAMATER_FILE, --paramater_file PARAMATER_FILE Input for standard definitions -i, --include_iostat_plots Include standard default iostat plots -m, --include_mgstat_plots Include standard mgstat plots -x, --output_csv_file output csv file -c CHART_FILE, --chart_file CHART_FILE Chart file definitions -o OUTPUT_DIR, --output_dir OUTPUT_DIR override output directory -l, --limit_yaxis limit y axis to 3 sigma maximum
One of the outputs of yaspe is an SQLite file with all the SystemPerformance metrics. e.g. yaspe_SystemPerformance.sqlite
Hint: Use default plots to see what it is you want to look at or deep dive in to.
If you already know what columns you care about, you can simply run yaspe to only create the sqlite file.
NOTE: Currently only supports Linux. Specifically;
- IRIS/Caché (mgstat)
- Linux (vmstat, iostat)
For example:
docker run -v "$(pwd)":/data --rm --name yaspe yaspe ./yaspe.py -i /data/your_html_file_name.html -x -a -c -s -o "yaspe"
Note: -x to include iostat. I used -a to skip the step of creating html charts in yaspe.
The resulting sqlite file name is derived from the -o parameter: yaspe_SystemPerformance.sqlite
You will need the disk /dev names for Database, Primary and Alternate Journal, WIJ, and Caché/IRIS disk.
They can all be the same device if that is how your system is set up. See the notes below for changing the defaults for charts.yml.
If you need clues to the device names look at the Linux info /dev/mapper section in the SystemPerformance html file.
For example; you can see below that the main IRIS database is on dm-7, journals are on dm-4 and dm-2, IRIS in this case is on dm-3, and so is the WIJ.

Below is an example using optional flags to start the zoom charts at 10:00 and end at 11:00:
docker run -v "$(pwd)":/data --rm --name yaspe yaspe ./pretty_performance.py -f /data/yaspe_SystemPerformance.sqlite -s 10:00 -e 11:00 -p /data/input.yml -i -m -c /data/charts.yml -o ./pretty_yaspe
Note: This example assumes:
cd to the folder with the sqlite file.charts.yml and input.yml, and they are in the same folder as the sqlite file (otherwise specify a path).-s and -e times!The following example shows an example of the input.yml file:
Site Name: "- Sample Site Name"
Disk List:
Database: "dm-7"
Primary Journal: "dm-2"
Alternate Journal: "dm-4"
WIJ: "dm-3"
IRIS: "dm-3"
Colormap Name: "Set1"
DPI: 300
WIDTH: 16
HEIGHT: 6
MEDIAN: False
Moving Average: 60
The following example shows an example of part of the charts.yml file:
Glorefs and vmstat:
Title: "Glorefs and vmstat"
columns_to_show:
column1: {"Text": "Glorefs", "Name": "Glorefs_mg", "axis": "left", "Style": "-", "Linewidth": 2, "Markerstyle": "", "Markersize": 1 }
column2: {"Text": "Total CPU", "Name": "Total CPU_vm", "axis": "right", "Style": "", "Linewidth": 2, "Markerstyle": "+", "Markersize": 3 }
zoom: False
y_label_l: "Global references/sec"
y_label_r: "Total CPU utilisation %"
y_max_l: 0
y_max_r: 100
Title : text appears in the title area of the chart, and is also used as part of the file name.
Columns to show : section lists each plot line, there is no hard limit on the number of lines (see Column names below).
column# : This section, one per plot line, lists pairs of keys with attributes. Attributes are:
zoom : if True, the chart x axis will limited to times specified in the command line time selection options -s and -e.
ylabel_l : The left hand side y label.
ylabel_r : The right hand side y label.
y_max_l : Maximum y axis left, e.g. 100 if you are showing %. 0 for max(). All charts start at 0.
y_max_r : Maximum y axis right.
Column names
Column names are derived from _mgstat, _vmstat, and the disk types in input.yml, for example;
_db is Database metrics columns._pri is Primary journal metrics._wij is WIJ metrics._mg is mgstat._vm is vmstat.Partial list, you should get the idea. Check out the .csv output for a full list:
datetime,rrqm/s_db,wrqm/s_db,r/s_db,w/s_db,rkB/s_db,wkB/s_db,avgrq-sz_db,avgqu-sz_db,await_db,r_await_db,w_await_db,svctm_db,%util_db,rrqm/s_pri,wrqm/s_pri,r/s_pri,w/s_pri,rkB/s_pri,wkB/s_pri,avgrq-sz_pri,avgqu-sz_pri,await_pri,r_await_pri,w_await_pri,svctm_pri,%util_pri,rrqm/s_wij,wrqm/s_wij,r/s_wij,w/s_wij,rkB/s_wij,wkB/s_wij,avgrq-sz_wij,avgqu-sz_wij,await_wij,r_await_wij,w_await_wij,svctm_wij,%util_wij,Glorefs_mg,RemGrefs_mg,GRratio_mg,PhyRds_mg,Rdratio_mg,Gloupds_mg,RemGupds_mg,Rourefs_mg,RemRrefs_mg,RouLaS_mg,RemRLaS_mg,PhyWrs_mg,WDQsz_mg,WDtmpq_mg,WDphase_mg,WIJwri_mg,RouCMs_mg,Jrnwrts_mg,ActECP_mg,Addblk_mg,PrgBufL_mg,PrgSrvR_mg,BytSnt_mg,BytRcd_mg,WDpass_mg,IJUcnt_mg,IJULock_mg,PPGrefs_mg,PPGupds_mg,r_vm,b_vm,swpd_vm,free_vm,buff_vm,cache_vm,si_vm,so_vm,bi_vm,bo_vm,in_vm,cs_vm,us_vm,sy_vm,id_vm,wa_vm,st_vm,Total CPU_vm
If you are using a local copy of yaspe you can run yaspe_runner.py.
Step 0. Add the folder with
yaspe.pyandyaspe_runner.pyto your PATH.

It is a common issue when trying to run graphical applications inside Docker containers on macOS.
Note this step requires a system restart!!
Ensure XQuartz is installed and running:
Settings > Security
To run the interface you need to do this in your terminal.
Start the container from the directory with the SystemPerformance HTML files, or a directory higher in the tree.
The HTML files will be in the /data directory or below (you will map"$(pwd)":/data) in thedocker runcommand.
# Allow connections to the X server xhost +localhostSet the DISPLAY environment variable
export DISPLAY=:0
Run the Docker container
docker run -v "$(pwd)":/data -v /tmp/.X11-unix:/tmp/.X11-unix -e DISPLAY=host.docker.internal:0 --rm --name yaspe yaspe ./yaspe_runner.py
The X11 screen will look like this:
