Chomsky & non-informativity

Informativity is linked to commitment. If the commitment is mendacious, mischievous, or wholly meretricious, it makes sense to minimize informativity so that detection and punishment is defeated. If a statement contains a novel category or referent, it may be subjectively informative even if  the statement does not have enough semantic structure to make it objectively informative. The same is true if its structure is incompossible, impredicative or involves an infinite regress. Chomsky's 'universal grammar' is non-informative because the rule 'break every grammatical rule' for any given language itself yields language. A 'universal grammar' is incompossible. It is no good saying that after humanity has ended, there would be a universal grammar describing every actual language. This is because every actual language could have been deformed to suit a Yoda who advertises non-humanness by violating grammar rules in a systematic manner. We understand Yoda because Yoda grammar is easy to 'decipher'. Indeed, with a little effort, we can even understand the fractured syntax of our leaders or our teenagers.

Grammar exists so as to increase the informativity of utterance. Ideally it would cash out as a unique structural causal model. However a non-informative utterance may be perfectly grammatically and an ungrammatical sentence may be more, not less, informative regarding strength of preference or intention.

Do Grammars have 'universal properties' in the category theory sense? Sure. If there is inter-translatability, that must be the case. But a set of 'universal properties' for grammars is not itself a universal grammar. What is? A set of rules which generates every other possible set of rules is a Von Neumann constructor of a certain type. But it isn't a generative grammar unless P=NP or there is an algorithmic way to differentiate randomness from non-randomness. Put another way, only if we could artificially produce life (takwin) indistinguishable from our own, could we say there was a Universal Grammar. This is not to say that 'at the end of Time' there may not be an infinity of such things. But the same could be said of unicorns and mermaids. Statements about them are non-informative. 

Theory or doctrine- 'matam' in Sanskrit- is non-informative. However every matam is linked to a vigyan, or science. If the vigyan is useful- well and good. But only if all relevant matams generate an observationally equivalent vigyan can we say they are dispensable. Maybe the vigyan can proceed without them. For a pious reason, they may be maintained as something supererogatory to Faith or eusebia.

On the other hand if a matam generates a different vigyan then there is a hypothesis of a potentially informative type based on what might not be a tautology of a premise. In this instance we may speak of competing Structural Causal Models. Utility will decide between them. They might fight back against this cruelty by getting metaphysical. But they can be crushed by a show of even greater stupidity and invocation of mumbo jumbo. Answer a fool according to his folly says the Good Book. But if the cunt has an expensive credential tell him he is a useless tosser. People are laughing at him. If he really were smart he'd be a billionaire by now. 

Chomsky says- It’s been understood forever that a theory has to answer two kinds of questions: Why this? Why not that?

This is nonsense. A theory has to make money or be fun or otherwise useful in some way to the cunt advancing it. Giving stupid answers to stupid questions may feature but again it may not. This is what has been understood forever.

There are two kinds of 'why this' questions. Protocol bound ones and time wasting shite you can ignore. Thus if the guy paying you asks 'why did you do this?' then there is a protocol bound method of answering because there are rules arising out of a contract and you need to show that the reason you did this rather than that was because it was what the rules of the contract required. If you can gain no benefit or avoid no penalty by answering a 'why this?' question, don't bother. 

Thinking or speaking may have no or few laws but thinking or speaking usefully can always be improved by the observance of protocols- even if they are rules you set for yourself.

On the other hand, one can have lots of protocols but if what one is doing is non-informative- it neither adds or substracts any facts worthy of consideration- then the thing is useless. 

We bother with grammar because we want to speak a particular language more correctly for a variety of utilitarian reasons. We must not bother with 'universal grammar' because it won't help us in any such way. This was not always obvious. Perhaps, it should have been. The truth is back in the Sixties, Computers were a bit magical. Maybe, by 2001 you'd have talking computers like Hal in Kubrik's film. Perhaps we'd have a 'universal translator' which would be like a hearing aid.  But that's not what actually happened is it? It is foolish to cling to this delusive hope.

Gary Marcus, in a post about Open AI's GPT-3, writes 

What is Chomsky on about? To really appreciate where he is coming from, you have to understand a bit of the history of linguistics, and his own role in developing the field.

Linguistics has always been around. The history of the shite Chomsky contributed to was that it was time wasting shite. It looked sciencey but it wasn't. It was stooooopid.  

Before Chomsky, most of what linguists did was to describe language, rather than to explain it;

Though to describe a thing to others is to explain as much of it as is currently explainable and to point to open question whereas to explain something of which no description exists is simply to masturbate. I can explain everything thanks to my theory of the Nicraguan horcrux of my neighbor's cat.  

one might, for example, characterize the grammar of English, or Japanese, or Swahili, perhaps focusing on some lesser known aspect of their grammar.

To say 'this is grammar' is to explain it. But so is saying 'aha! a clear manifestation of the malignity of the Nicaraguan horcrux of my neighbor's cat!'  

Chomsky found that description, in of itself, dissatisfying, and wanted to ask a different question: why was human language the way that it is?

Because of the universal grammar or the Nicaraguan horcrux or whatever bee you have in your bonnet. The problem here is that the we don't the way human language actually is. If we did, we'd know how to write jokes which everybody would find funny and our leaders could make speeches which everybody would find inspiring. It may be that some people can do this sort of thing already. But is there an algorithm for it? We don't know. There is some point to describing languages better and better. There is no point to pretending one has an explanation for why language, or pudding, or Nicaraguan horcruxes, are they way they are. 

Why do we speak in human languages, rather than say programming languages?

Because we are human beings, not computers which have been programmed.  

Why are there commonalities across many different languages?

Because there are commonalities across human beings.  

(Chomsky’s hypothesis: all humans, regardless of culture, acquire language in way that is governed by universal grammar.)

only in the sense that universal grammar is governed by the Nicarauguan horcrux of my neighbor's cat. The fact is that there are an infinity of unique universal grammars but none are accessible.  It would be easy to take any posited universal grammar and quickly find a counterexample- a sentence which is meaningful but which doesn't obey its rules. 

It’s one thing, Chomsky argued, to catalog all the world’s flowers, and another to try to understand what it is about biology that makes us have the flowers that we do have rather than something entirely different.

Indeed. The first exercise is useful. Horticulture is a big business. The second exercise- unless you are an evolutionary business is useless though no doubt artists can make money from thinking otherwise useless thoughts. But it is some special skill of theirs, not the thoughts themselves, which create value.  

It is one thing to try to understand why we have what we have rather than something different. But it is another thing to understand that we don't know what we have and wouldn't know if something different from what we have suddenly appeared. Science begins with this understanding. 

Same thing with language: catalog, versus explanation.

Something useful versus something utterly useless.  

Chomsky spent a large part of his career trying to get people to think about why human language is the way that it is.

He spent a large part of his career trying to get people to see the world in the way he sees it. They refused because it was pointless and actively mischievous. Meanwhile, genuine scientists were doing useful stuff. Useful, not messianic. 

The why question, is to be sure, incredibly difficult,

No it isn't. The answer is obviously the Nicraraguan horcrux of my neighbor's cat.  A Guatemalan horcrux would be absurd.  

in part because no other extant species has anything remotely like it,

There's no reason our species shouldn't branch out such that interbreeding becomes impossible.  Indeed, people might start altering their DNA and so you'd have dolphin people and horse people and so forth. 

and in part because we don’t (and shouldn’t) do invasive studies in which we carve open living human brains or (e.g.) teach humans artificial languages in lieu of natural languages in order to better understand the underlying mechanisms.

Neurosurgeons may have to keep a patient conscious and talking while doing brain surgery. An artificial language taught to a kid is still a natural language- though it may resemble a pidgin. If the kid has companions, they will develop a creole just as sophisticated as any other language. 

But the question of why human language is as it is remains one of the most important questions a scientist could ask:

The answer has to do with evolution. Cataloguing and describing can give us laws- e.g. Grimm's law- and to decide the relative antiquity of ancient inscriptions. Chomsky didn't bother with things like that. He his wasted time trying to shit higher than his arsehole.   

humans are truly unique, and their signature trait is the way that they turn sound (or gesture) into meaning, enabling culture and technology transmission at unprecedented scale.

Because it useful to do so.  

From the perspective of science, it would be a tremendous advance if we could understand how humans come to do that, what machinery enables children to do what our best computers still can’t do, and how the capacity to learn language evolved. (It would likely also have considerable payoff for engineering, but that’s a story for another day.)

We have Structural Causal Models for this. But the tech is a work in progress. 'Universal Grammar' won't help any.  

If Chomsky’s more than a little disappointed about by the attention surrounding systems like GPT-3, it’s because all this emphasis on statistical machines has distracted away from the incredibly important questions about the very nature of human language and cognition that he spent his career rightly trying to call attention to.

But nothing came of that attention. Something has come from the statistical, 'e-language' approach. Google translate is actually quite helpful.  

His questions still deserve answers, and it is difficult to see how GPT-3 casts any light on them.

No light can be cast on nonsense.  

Large language models can, parasitically, glom onto human data, but they could glom onto any other data set just as well (e.g. programming languages);

which is why they are useful 

as such, they tell us far too little about how humans have come to be what they are.

That matters.

Not if you are concerned with utility. Suppose you desperately need to take a dump. You ask a guy where the nearest toilet is. He points the way. That is useful. True, it doesn't tell you very much about how humans have come to be what they are. But it would have annoyed the shit out of you if it had done so, instead of pointing to the place where you could take your dump rather than end up with fudging your pants.