Data Engineer

The ML team needed clean training data, but it was buried in 15,639 SharePoint files with no consistent structure. I built an authenticated PowerShell pipeline through an Azure App Registration to pull every file and sort it into six welding-process categories, and tuned it well enough that 70% needed no human review at all before it went to the team for fine-tuning.

The part I'm proudest of isn't the classifier — it's the logging. Every run wrote structured CSVs at both the company and the file level, so when something looked wrong you could trace exactly what happened instead of re-running blind. Across the parallel letter-batch runs, that auditability saved an estimated $15,000+ in engineer time.

The other half was moving a live database off legacy Linode MySQL onto managed DigitalOcean — 29,000+ rows across 87 tables — without dropping or duplicating anything. The tables had foreign-key dependencies, so insert order mattered: I wrote a topological sort to resolve them, made the inserts idempotent and batched so a failed run could be retried safely, and validated primary keys symmetrically before and after. Every row was accounted for on both sides — 100% parity.