Скачать или смотреть WOVE: INCORPORATING WORD ORDER IN GLOVE WORD EMBEDDINGS

WOVE: INCORPORATING WORD ORDER IN GLOVE WORD EMBEDDINGS
Mohammed Ibrahim University of Arkansas, United States of America / Susan Gauch University of Arkansas, United States of America / Tyler Gerth University of Arkansas, United States of America / Brandon Cox University of Arkansas, United States of America

(Presented at the International Conference on Humanities, Social and Education Sciences (IHSES)) which took place on April 22-25, 2021, New York, USA, https://www.2021.ihses.net/ organized by International Society for Technology, Education and Science ((ISTES)-http://www.istes.org)

Word vector representations open up new opportunities to extract useful
information from unstructured text. Defining a word as a vector made it easy for the machine learning algorithms to understand a text and extract information from. Word vector representations have been used in many applications such word synonyms, word analogy, syntactic parsing, and many others. GloVe, based on word contexts and matrix vectorization, is an effective vector-learning algorithm. It improves on previous vector-learning algorithms. However, the GloVe model fails to explicitly consider the order in which words appear within their contexts. In this paper, multiple methods of incorporating word order in GloVe word embeddings are proposed. Experimental results show that our Word Order Vector (WOVe) word
embeddings approach outperforms unmodified GloVe on the natural language tasks of analogy completion and word similarity. WOVe with direct concatenation slightly outperformed GloVe on the word similarity task, increasing average rank by 2%. However, it greatly improved on the GloVe baseline on a word analogy task, achieving an average 36.34% improvement in accuracy.