Скачать или смотреть Domain Specific Word Segmentation and Hierarchy Detection using NLP Algorithm By Genpact [MLDS2020]

Machine Learning Developers Summit 2020
https://mlds.analyticsindiasummit.com/
Computational Linguistics or the study of Natural Language Processing is the modern approach for a machine to understand and interpret language including its grammar, semantics, phonetics etc. by using large datasets and computational tools and techniques.

Back in the 1990s, statistical machine learning methods began to replace the classical top-down rule-based approach to interpret languages, primarily due to accurate results, speed of processing and robustness of the algorithms.

The advent of faster processors in the 2010s exponentially improved the performance of NLP algorithms. The need for clean and labeled training data also grew along with the emergence of faster processors and robust algorithms to improve the accuracy of the models. Statistical approaches have turned another corner and are now strongly focused on the usage of deep neural networks to both perform inferences on specific linguistic tasks and for developing robust algorithms.

This paper By Abhishek Bhadra AVP at Genpact, Prakash Selvakumar AVP at Genpact, focuses on a machine learning approach to perform word segmentation and Hierarchy detection on medical documents (in form of any editable digital documents like PDFs).

When text is extracted from digital pdf using python libraries such as PDFMiner, the output is in the form of Scriptio continua - a style of writing without spaces between the words or sentences. One of the ways to use this unseparated raw text in a meaningful way is known as word segmentation, which is a process to determine the word boundaries in a sentence.

In this paper, they present a machine learning approach to build a word segmentation algorithm and also find the hierarchical structure in the text (for example Header1 or Header2 etc.). They leveraged the English Wikipedia dataset to build and train this advanced sequence to sequence model.

The technique used in this paper has been successfully tested on medical domain data. We have also observed significant improvement in model accuracy by further training the algorithm with domain-specific documents (in the form of PDFs)

___________

Abhishek Bhadra is a seasoned Analytics expert and Bilingual AI/ML Practitioner. His distinguished career spans more than 14 years with a proven track record in driving disruption through augmented analytics across multiple Fortune 500 companies. As one of the Analytics & Consulting leaders in Genpact, Abhishek has driven multiple high impact client engagements, by leveraging digital and advanced analytics solutions to transform core business processes. In the last 14 years, Abhishek had taken up multiple leadership roles within Genpact Analytics, ranging from leading a 100+ member Analytics Delivery team to spearheading Transformation & Solutions for multiple Fortune 500 clients across various industries (Aviation, Automotive, Telecom, Banking, CPG etc.). Abhishek is also a visiting Guest lecturer in some of the Top Indian Business Schools and has published multiple articles in international journals.