Jump to content

Distributional semantics

From Wikipedia, the free encyclopedia
This is an old revision of this page, as edited by Jeran Renz (talk | contribs) at 15:35, 13 February 2020 (→Distributional hypothesis: +precision). The present address (URL) is a permanent link to this revision, which may differ significantly from the current revision.

Distributional semantics is a research area that develops and studies theories and methods for quantifying and categorizing semantic similarities between linguistic items based on their distributional properties in large samples of language data. The basic idea of distributional semantics can be summed up in the so-called Distributional hypothesis: linguistic items with similar distributions have similar meanings.

Distributional hypothesis

The distributional hypothesis in linguistics is derived from the semantic theory of language usage, i.e. words that are used and occur in the same contexts tend to purport similar meanings.[1]

The underlying idea that "a word is characterized by the company it keeps" was popularized by Firth in the 1950s.[2]

The distributional hypothesis is the basis for statistical semantics. Although the Distributional Hypothesis originated in linguistics,[3] it is now receiving attention in cognitive science especially regarding the context of word use.[4]

In recent years, the distributional hypothesis has provided the basis for the theory of similarity-based generalization in language learning: the idea that children can figure out how to use words they've rarely encountered before by generalizing about their use from distributions of similar words.[5][6]

The distributional hypothesis suggests that the more semantically similar two words are, the more distributionally similar they will be in turn, and thus the more that they will tend to occur in similar linguistic contexts.

Whether or not this suggestion holds has significant implications for both the data-sparsity problem in computational modeling,[7] and for the question of how children are able to learn language so rapidly given relatively impoverished input (this is also known as the problem of the poverty of the stimulus).

Distributional semantic modeling

Distributional semantics favor the use of linear algebra as computational tool and representational framework. The basic approach is to collect distributional information in high-dimensional vectors, and to define distributional/semantic similarity in terms of vector similarity.[8] Different kinds of similarities can be extracted depending on which type of distributional information is used to collect the vectors: topical similarities can be extracted by populating the vectors with information on which text regions the linguistic items occur in; paradigmatic similarities can be extracted by populating the vectors with information on which other linguistic items the items co-occur with. Note that the latter type of vectors can also be used to extract syntagmatic similarities by looking at the individual vector components.

The basic idea of a correlation between distributional and semantic similarity can be operationalized in many different ways. There is a rich variety of computational models implementing distributional semantics, including latent semantic analysis (LSA),[9][10] Hyperspace Analogue to Language (HAL), syntax- or dependency-based models,[11] random indexing, semantic folding[12] and various variants of the topic model.[13]

Distributional semantic models differ primarily with respect to the following parameters:

Distributional semantic models that use linguistic items as context have also been referred to as word space, or vector space models.[15][16]

Compositional

Compositional distributional semantic models are an extension of distributional semantic models that characterize the semantics of entire phrases or sentences. This is achieved by composing the distributional representations of the words that sentences contain. Different approaches to composition have been explored, and are under discussion at established workshops such as SemEval.[17]

Simpler non-compositional models fail to capture the semantics of larger linguistic units as they ignore grammatical structure and logical words, which are crucial for their understanding.[citation needed]

Applications

Distributional semantic models have been applied successfully to the following tasks:

Software

See also

People

References

  1. ^ Harris 1954
  2. ^ Firth 1957
  3. ^ Sahlgren 2008
  4. ^ McDonald & Ramscar 2001
  5. ^ Gleitman 2002
  6. ^ Yarlett 2008
  7. ^ Wishart, Ryder, and Prokopis Prokopidis. “Topic Modelling Experiments on Hellenistic Corpora.” In Proceedings of the Workshop on Corpora in the Digital Humanities 17, 39–47. Bloomington, IN: CEUR Workshop Proceedings, 2017, Online: https://pdfs.semanticscholar.org/bd71/ab40960e481006117bafd0ae952d3e8d1f66.pdf.
  8. ^ Rieger 1991
  9. ^ Deerwester et al. 1990
  10. ^ Landauer, Thomas K.; Dumais, Susan T. (1997). "A solution to Plato's problem: The latent semantic analysis theory of acquisition, induction, and representation of knowledge". Psychological Review. 104 (2): 211–240. doi:10.1037/0033-295x.104.2.211. ISSN 1939-1471.
  11. ^ Padó & Lapata 2007
  12. ^ De Sousa Webber, Francisco (2015). "Semantic Folding Theory And its Application in Semantic Fingerprinting". arXiv:1511.08855 [cs.AI]. {{cite arXiv}}: Cite has empty unknown parameters: |access-date= and |publisher= (help)
  13. ^ Jordan, Michael I.; Ng, Andrew Y.; Blei, David M. (2003). "Latent Dirichlet Allocation". Journal of Machine Learning Research. 3 (Jan): 993–1022. ISSN 1533-7928.
  14. ^ Church, Kenneth Ward; Hanks, Patrick (1989). "Word association norms, mutual information, and lexicography". Proceedings of the 27th Annual Meeting on Association for Computational Linguistics -. Morristown, NJ, USA: Association for Computational Linguistics: 76–83. doi:10.3115/981623.981633.
  15. ^ Schütze 1993
  16. ^ Sahlgren 2006
  17. ^ "SemEval-2014, Task 1".

Sources

Retrieved from "https://en.wikipedia.org/w/index.php?title=Distributional_semantics&oldid=940609519"