Word Vectors

Word vectors are vectors of numbers that represent a word’s meaning. Words are mapped to a multidimensional geometric space using continuous floating-point numbers. Words with semantic relations appear closer to each other within this space.

Understanding words in this manner allows for the use of mathematical operations on text such as “king – man + woman = queen.”

The vector space model was developed in the 1960s for information retrieval. Singular value decomposition reduced the number of necessary dimensions, paving the way for latent semantic analysis.

There are different ways of generating this mapping, including neural networks (word2vec) and probability models (counting co-occurrences). A challenge of the vector space model is that all possible meanings of a word are combined into one representation.