[PLing] Talk by Thomas Graf: Linguistics and Symbolic Computation in a World of Large Language Models (2023-12-20)

[Tristan Miller]

Given the rapid rise of large language models, will symbolic linguistics 
be left in the dust, or is this actually an opportunity for meaningful 
synergy between symbolic and subsymbolic approaches? This question is 
addressed in "Linguistics and Symbolic Computation in a World of Large 
Language Models", a talk by Thomas Graf of Stony Brook University. The 
talk is part of the 2023 Fall Lecture Series of the Austrian Research 
Institute for Artificial Intelligence (OFAI).

Members of the public are cordially invited to attend the talk via Zoom 
on Wednesday, 20 December 2023 at 18:30 CET (UTC+1):

URL: 
https://us06web.zoom.us/j/84282442460?pwd=NHVhQnJXOVdZTWtNcWNRQllaQWFnQT09
Meeting ID: 842 8244 2460
Passcode: 678868

You can add this event to your calendar: 
https://www.ofai.at/calendar/2023-12-20graf.ics

Talk abstract: Language has always played a central role in artificial 
intelligence, yet AI researchers and linguists have rarely seen eye to 
eye on things, in particular the status of subsymbolic/neural approaches 
to language. After decades of debates, it looks like the subsymbolic 
approaches have finally emerged victorious. Not only are large language 
models (LLMs) succeeding in incredibly complex real-world tasks, 
subsymbolic models are also rapidly gaining traction in some areas of 
theoretical linguistics, e.g. lexical semantics. This raises the 
question: will symbolic linguistics be left in the dust, or is this 
actually an opportunity for meaningful synergy between symbolic and 
subsymbolic approaches?

In this talk, I argue for the latter by presenting “subregular syntax” 
as a concrete example of what such a synergy may look like. Subregular 
syntax is a symbolic approach that combines formal language theory with 
the Minimalist syntax framework proposed by Noam Chomsky, which grants 
it a large degree of empirical coverage across a wide range of 
typologically diverse languages. Despite that broad coverage, subregular 
syntax is a very simple formalism that analyzes all syntactic 
dependencies in terms of relativized adjacency conditions. Even though 
these conditions are stated over trees, they can actually be reduced to 
a very specific types of n-grams over strings. This opens up a new way 
of representing sentence structure in neural networks while bringing 
robust learning algorithms like stochastic gradient descent to 
Minimalist syntax. It also casts doubt on claims in the literature that 
the behavior of neural networks in specific linguistic tasks, e.g. 
binding or NPI-licensing, shows that they use tree structure. Instead, 
these findings may be indicative of a network’s ability to use fairly 
elaborate types of n-grams. The careful study of the symbolic approach 
of subregular syntax thus is an opportunity to deepen our understanding 
of neural networks while also harnessing their advantages for 
theoretical linguistics.

Speaker biography: Thomas Graf is Associate Professor of Computational 
Linguistics in the Department of Linguistics at Stony Brook University. 
He also holds an affiliate appointment in Stony Brook’s Institute for 
Advanced Computational Science. Before joining Stony Brook, he studied 
linguistics at the University of Vienna and received his PhD from UCLA 
in 2013. His research operates at the intersection of theoretical 
linguistics, computational linguistics, and cognitive science, with a 
particular focus on syntax (sentence structure). He is the recipient of 
the 2014 E.W. Beth Dissertation Prize for outstanding PhD theses in 
logic, language, and information, and in 2019 he received an NSF CAREER 
award for his research on subregular syntax. He is the creator of the 
blog Outdex, which covers computational and theoretical linguistics, and 
he loves to introduce high school students to the wonders of 
computational linguistics during Stony Brook’s Mathematics Summer Program.


-- 
Dr.-Ing. Tristan Miller, Research Scientist
Austrian Research Institute for Artificial Intelligence (OFAI)
Freyung 6/6, 1010 Vienna, Austria | Tel: +43 1 5336112 12
https://logological.org/ | https://punderstanding.ofai.at/