My research is focused on how humans and machines process and learn natural languages. I'm particularly interested in the issue of generalisation, that is, of how we make the best use of limited training data to handle novel inputs effectively, and of how this issue relates to traditional linguistic topics such as syntactic structure and semantic composition.

My PhD (https://era.ed.ac.uk/handle/1842/4927) looked at how to combine vectors representing the meaning of individual words to make representations of larger phrases and sentences. I still think this question of how to handle semantic composition is one of the most important in AI. The deep learning answer is that composition is some non-linear function that you learn in the hidden layers of a large language model, but that's a little bit more vague than I would like.

In particular, the handling of structure has typically been a lot less sophisticated in neural network approaches than in more traditional symbolic approaches. I'm interested in how this can be addressed and I've looked at various questions relating to syntax (e.g. https://ccneuro.org/2019/proceedings/0000188.pdf) and particularly how to disentangle semantic and syntactic information (e.g. https://aclanthology.org/P15-1126.pdf).

Chomsky introduced the idea of characterising the structure of languages in terms of a computational hierarchy, leading to natural languages being characterised as requiring at least a weakly context sensitive grammar. This question of how much flexibility versus constraint should be incorporated into our models is important for two reasons. From the point of view of language acquisition, learning in an overly expressive formalism will be slow and data hungry. And from the point of view of science, narrower hypotheses are more readily falsifiable and therefore carry more empirical content. In the limit, a hypothesis space that contains all non-linear functions is entirely vacuous.

I think, therefore, that the investigation of the innate biases already present in our current models and those needed in future models is really interesting. One way to think about these issues is in terms of transformations: our models should generalise effectively under certain sorts of transformation (e.g. reversing a contradiction: https://arxiv.org/pdf/1805.06648.pdf) but should fail to even learn under certain other transformations (e.g. reversing the end of a sentence: https://aclanthology.org/2020.coling-main.451.pdf).