MoL-2018-13: Recognizing Logical Entailment: Reasoning with Recursive and Recurrent Neural Networks

MoL-2018-13: Mul, Mathijs S. (2018) Recognizing Logical Entailment: Reasoning with Recursive and Recurrent Neural Networks. [Report]

[thumbnail of MoL-2018-13.text.pdf]

Download (9MB) | Preview


This thesis studies the ability of several compositional distributional models to recognize logical entailment relations. We first investigate the performance of recursive neural matrix and tensor networks on an artificially generated data set labelled according to a natural logic calculus. Several issues with this set-up are identified, which we aim to solve by introducing a new task: First-Order Entailment Recognition. In combination with an automated theorem prover and model builder, an artificial language with higher grammatical complexity is used to generate a new data set, whose labels are determined by the semantics of first-order logic. The tree-shaped networks perform well on the new task, as opposed to a bag-of-words baseline. Qualitative analysis is performed to reveal meaningful clusters on the level of word embeddings and sentence vectors. A novel recurrent architecture is proposed and evaluated on the same task. The high testing scores obtained by GRU cells in particular prove that recurrent models can learn to apply first-order semantics without any cues about syntactic structure or lexicalization. Projection of sentence vectors demonstrates that negation creates a mirroring effect at sentence level, while strong clustering with respect to verbs and object quantifiers is observed. Diagnostic classifiers are used for quantitative interpretation, which suggests that the best-performing GRU encodes information about several linguistic hypotheses. Additional experiments are conducted to assess whether the recurrent models owe their success to compositional learning. They do not profit from the availability of syntactic cues, but prove capable of generalization to unseen lengths. After training with fixed GloVe vectors, the GRU can handle sentence pairs with unseen words whose embeddings are provided. These results suggest that recurrent models possess at least basic compositional skills.

Item Type: Report
Report Nr: MoL-2018-13
Series Name: Master of Logic Thesis (MoL) Series
Year: 2018
Subjects: Computation
Depositing User: Dr Marco Vervoort
Date Deposited: 11 Sep 2018 12:16
Last Modified: 11 Sep 2018 12:16

Actions (login required)

View Item View Item