The project called for building a program to parse Chinese text in a MySQL database. Each row had a content column consisting of many paragraphs of Chinese text. My solution segmented the text and stored the result in a 2nd column, then listed unique nouns and stored that list in a 3rd column. In successor projects I continually refined the performance of the algorithm to run more efficiently across the millions of rows in the client's database. From the initial runtime of 19 hours I eventually got it down to 2.5 hours.
Built with
- Python
- MySQL