I worked for more than 30 years on these systems and they could never work. I implemented a very fast NLP symbolic parser, for which the team I worked with created grammars for 8 languages, including Japanese. In 2007, with a grammar of 60,000 rules, we could parse at a speed of 3000 words/s (see https://github.com/clauderouxster/XIP for the Open Source version). The parser could extract syntactic dependencies, and could use ontologies. But language is like sand, the more you try to grab, the more you leak. There was a kind of futility in trying to compress languages into rules, nothing actually scaled up. Still, we managed to win competitions as late as 2016 with SemEval sentiment analysis, and in 2017, we also ranked first in a legal document extraction campaign organized by IBM, but to no avail. It was a lot of work, and the conclusion was very simple. We had to push our grammars as far as possible into lexical grammars, which eventually LMM managed to really implement. We discovered very early, that context was all that mattered. We tried to create grammars that would apply to a full paragraph instead of sentences, but then the performance would plummet. The reason why LLM work, is that at each step they compress the whole context into a meaningful vector, which they then used to guide the rest of the generation process. I spent my whole life in the pursuit of a perfect parser with very brillant people, and I really find hard to say that not only did we fail, but that LLM is the response we were looking for.

~ claudius