Automating Technical Text Classification - Failure mode classification for reliability engineers

Failure mode classification (FMC) is a task performed by reliability engineers that captures and codes failure and other undesirable events.
These codes are used to prioritise improvement work and update maintenance strategies. It is a difficult task to perform manually as the observations of each event are captured by field technicians in natural language. This talk describes a pipeline for automating FMC using open source technical language processing tools developed by the NLP-TLP group at the University of Western Australia. We compare approaches using a traditional language model trained with annotated data to out-of-the-box Chat GPT-3.5 and Chat GPT-3.5 model fine tuned (FT) on annotated data. The GPY-3.5(FT) model shows very promising performance. We conclude with some observations on the use of these approaches in practice based on our work with industry partners.

A paper describing this talk in more detail is available at https://arxiv.org/abs/2309.08181

This is a recording of a talk given to industry at the UWA Data Institute's data.conversation event in August 2024.