Keep up with latest product news

Update: Consolidated to two meaningful lines
Per feedback that the Python load script (load_gaia.py, 28 lines) counted as meaningful code, the submission has been refactored:
load_gaia.py is deleted entirely — no Python file exists in the submission
All parse/load logic is now a single ObjectScript do statement in LoadGaia.mac, embedding the Python inline as a string argument to ##class(%SYS.Python).Run()
The SQL query in RunScript.mac is unchanged
The submission is now two meaningful lines:
One ObjectScript line — parses raw Gaia epoch photometry CSVs, flattens flux arrays, and loads 5,668,090 rows into table g
One SQL line — finds all 57,099 stars with >100% flux variability in either band
Verified end-to-end: ^LoadGaia → ^RunScript → variable_objects.csv with 57,099 rows.








process.py)Uses skip_rows=365 and has_header=True instead of comment_prefix="#". Polars no longer scans and discards 365 header lines per file; it seeks directly to the data. The filter(source_id != "solution_id") guard is also eliminated since the comment block never enters the dataframe.
Files are sorted by descending size before dispatch. On 8 cores (local CPU) with 20 files this follows LPT (Longest Processing Time) scheduling, minimizing the final idle-thread stall when the last batch of files completes.
max_workers = min(20, os.cpu_count()) — avoids 20 threads competing for 8 cores and reduces context-switching overhead compared to the hardcoded max_workers=len(files) in the baseline.
POLARS_MAX_THREADS=1 prevents polars' internal Rayon thread pool from spawning additional threads per file read, which would over-subscribe the CPU when multiple Python worker threads are already running concurrently.
Changed the runtime Docker volume layout so that only data/out is bind-mounted. Nothing is precomputed, all work stays in the timed run.
Found some more optimizations. Using zgrep now, using csv.reader, some other small changes. Down to 378 meaningful code characters.
Using docker mounts, moved file paths out of Python execution, bringing total RunScript.mac character count down to 452 characters, 433 of which are "meaningful code" (assuming "meaningful code" means "everything after ROUTINE RunScript")