Professional workClosed source · In production

Coal Models Migration

Migrating MATLAB models to Python on Databricks.

I architected the full pipeline and created the final production implementation for the coal models migration, moving a legacy MATLAB system to Python on Databricks. This work was part of the same broader modernization effort as the Natural Gas Liquids Daily Supply Models migration.

01 / The Challenge

The coal modeling system needed to move from a legacy MATLAB environment into Python on Databricks as part of a broader modernization program.

The project required a complete pipeline design and a production implementation, extending beyond a language translation.

System view

01Source data

02Airflow

03Databricks notebooks

04PySpark models & transformations

05Published coal model data

02 / The Approach

Architected the full coal pipeline: Airflow orchestrates Databricks notebooks that run PySpark models and transformations, then publish coal model data.
Migrated the legacy MATLAB code to Python and created the final implementation used in production.
Used Genie as the primary code-generation tool during implementation, while owning the pipeline architecture and final production code.
Deployed the completed modeling pipeline to production.

03 / The Impact

Moved the coal models from the legacy MATLAB system to Python on Databricks.
Delivered the full production pipeline as part of the same modernization effort as the NGL models.