Sen CutlerData Engineer

Fitting a Distribution-Mapping Model to Convert a 4,000-Record Corpus

Two reference corpora described the same domain under incompatible specifications. No crosswalk existed, and the attribute scales were too dissimilar for hand-written rules. I extracted both from scanned PDFs, loaded them into PostgreSQL, and normalized the data. I then fit a model based on percentile mapping: I built empirical distributions for every numeric attribute on both sides, and an algorithm to map records from the legacy corpus to the modern one. Conversion encodes a source record's parameters as a percentile vector, then decodes it against the target distribution. Descriptive text went through constrained LLM prompting and was parsed back to JSON. The deliverable is a CLI client with lookup, search, and several render formats.

  • Python
  • PostgreSQL
  • Statistical Modeling
  • Prompt Engineering
  • PDF

AI · Machine Learning · Data Quality