Скачать или смотреть The General Index

The General Index consists of 3 tables derived from 107,233,728 journal articles. A table of n-grams, ranging from unigrams to 5-grams, is extracted using SpaCy. Each of the 355,279,820,087 rows of the n-gram table consists of an n-gram coupled with a journal article id. A second table is constructed using Yake and consists of 19,740,906,314 rows, each with a keywords and an article id. A third table associates an article id with metadata.

The metadata, readme, and sample files are available on this item in the data downloads area.
The README.txt file contains more information.
The corpus of articles has been split into 16 slices. We keep the keywords and n-grams files, one for each of the 16 slices, in separate items. We do this to avoid overloading the servers. Be careful, these files are big and inflate greatly upon unzipping.
The keywords are located here: [ 0 : 1 : 2 : 3 : 4 : 5 : 6 : 7 : 8 : 9 : a : b : c : d : e : f ]
The n-grams are located here: [ 0 : 1 : 2 : 3 : 4 : 5 : 6 : 7 : 8 : 9 : a : b : c : d : e : f ]