historically, "intelligence" denotes a social concept founded in colonial preconceptions of the inherent superiority of the white male. You might think that's a stretch, but the entire history is race science and IQ test calibrations were tweaked when they gave unacceptable outputs like women or Kenyans coming out smarter.

the more I look, the more corrupt the entire idea complex denoted by "intelligence" is, in theory and practice.

it's not a coincidence that the artificial intelligence endeavour is also riddled with blatant race scientists

@claralistensprechen5th yeah, to be clear, yes|no|maybe is a gross abstraction and oversimplification. Brains and thought aren't boolean things. If they were, there would be no room for uncertainty!

But we really have no idea how many possible states there are exactly, other than "more than two." Could be 3, could be 30,000. (The 30k is way more likely.)

You appear to be confusing Knightian uncertainty for risk. Risk is something that can be quantified. It's what insurance companies trade in. Knightian uncertainty cannot be quantified. Since you are putting aside defining what this "risk" is supposed to be about ("you don't need to define intelligence"), then it cannot possibly be a "very real risk". For one thing because it literally is not real!

Granted, most of the AI shills and xrisk barkers out there are either ignorantly or purposely conflating risk and (Knightian) uncertainty, probably because it serves their cause and makes their sci fi stories sound more exciting. No one is going to jump on the "We have no clue what intelligence is or how to synthesize it or whether that would be a dangerous thing to do. In fact we don't even know when we might know any of that" bandwagon. So confusion is understandable.

@davidgerard@circumstances.run

@abucci Maybe I'm confusing terms, but keep in mind that you don't know what you don't know. I've been interested about AI safety long before the AI craze. I don't think that transformer based LLMs will pose a risk in the future, but LLMs (possibly of a different architecture) will be a piece of the puzzle of a system that we possibly could not control. And it's a very different type of risk. Most other risks are recoverable. A nuclear catastrophe? Society will still exist and it will recover. A runaway AI that gets into power and we cannot control? It's game over. At the very least, life long dictators end up dying of old age. But having a dictator that can clone itself is a different story.

You don't need a conscience in a machine that is capable of manipulating people to reach the keys of the realm.

a piece of the puzzle of a system that we possibly could not control. And it's a very different type of risk.

I'll say it again: it's not a risk if you cannot quantify it. That's what the word "risk" is usually taken to mean.

When you say such and such is something that we possibly could not control, what's your basis? I assert that we possibly could control it. End of conversation forever, because it boils down to nothing more than a clash of two meaningless, evidence-free assertions that obligate no one to assign any weight to. Another "risk" of this nature is that a jar of mustard in your fridge goes out of control and kills everyone. This "risk" has exactly the same weight as the "risk" that "AI" does. You don't see this?

Nyquist doesn’t really address the objection. It shows that certain continuous signals can be reconstructed from discrete samples under strict conditions. But the claim here isn’t about bandlimited signals — it’s about cognition and thought. Human intelligence seems to involve structures that aren’t reducible to neatly sampleable functions, especially when you bring in infinities, self-reference, and biochemical dynamics.

Approximations can be powerful, but approximation is not identity. A Bézier control net isn’t the curve. Saying “for practical purposes” is a move into engineering sufficiency, not a rebuttal of the point that digital computers may be ontologically limited in ways brains are not.