The client's business depends on procurement done between agents and other companies' salespeople using Skype chat. They wanted data on the offers made by the salespeople and brought me on to write a program that could parse the semi-structured chat data into a financial spreadsheet.
They exported a combined chatlog as a 190 MB JSON file. I wrote a parser to locate the sales offers and extract them, using ~2.5 Kb of regular expressions. It also loaded the official datasheet for all permitted pharmaceuticals in the U.S. and collated the two. The output was a combined spreadsheet of over 10,000 offers going back 1½ years which provided key insights. I delivered the program, which they continued to use. I was later called on to improve its effectiveness, which I did by adding a second parsing layer written as a context-free grammar.
Built with
- Data Mining
- Python
- Data Cleaning
- pandas