On Distributed Representation in Word Semantics

The dualism of the rationalistic tradition of thought is sketched in view of the "semiotic problem" of meaning constitution. Being a process of cognition which is based upon communicative interaction by signs, their usages (in linear order and selective combination), constitute language structures. Other than "symbolic" representational formats employed so far in natural language processing by machine, it is argued here that "distributional" representations correspond directly to the way word meanings are constituted and understood (as fuzzy structures of world knowledge) by (natural and artificial) information processing systems. Based upon such systems' theoretical performance in general and the pragmatics of communicative interaction by real language users in particular, the notions of "situation" and "language game" as introduced by Barwise/Perry and Wittgenstein respectively are combined to allow for a numerical reconstruction of processes that simulate the constitution of meaning and the interpretation of signs. This is achieved by modeling the linear or "syntagmatic" and selective or "paradigmatic" constraints which natural language structure imposes on the formation of (strings of) linguistic entities. A formalism, a related algorithm, and test results of its implementation are given in order to substantiate the claim for an artificial "cognitive information processing system" (CIPS) that operates in a linguistic environment as some meaning acquisition and understanding device.

ICSI Technical Report TR-91-012

Technical Report