Yeah…
I’m reading this article
https://link.springer.com/article/10.1007/s13347-023-00621-y
And I like it because it conforms to my biases - ChatGPT has no cognitive process, it cannot evaluate the semantic content of the text it produces (which I would argue means there is no semantic content. We interpret the text as having meaning, but in truth it is meaningless, pure noise that simply resembles speech and is sometimes “correct” by accident) and it’s basically a neat magic trick.
Also, turns out that ChatGPTs miraculous abilities are partially due to workers in the Kenya who were paid a pittance to read the most awful content imaginable and label it. CW: Mention of CSA, SA, violence, trauma, worker exploitation. The man-made horrors are well within my comprehension
According to the article the company was also doing content moderation for Facebook for a while but pulled out because doing content moderation for facebook is a one-way ticket to horrific trauma and it doesn’t take very long to get there.
Shiny cool new toys and gross violations of ethics and human dignity foisted on test subjects in Africa. Name a better combination.
One massive issue I have with trying to make sense of any of this; I don’t know enough about how these machines work to assess whether a given article is written by someone capable of giving informed opinions, or if it’s written by a bazinga true believer high on their own supply. This is especially problematic with “AI” language models; Bazingas may not actually know a goddamn thing about neuroscience, linguistics, or semantics and are just failing the Turing Test left and right attributing human like attributes to a weighted random number generator. And the linguists et al may not understand what’s going on on the math side. Love me some known unknowns and unknown unknowns. This is why I usually don’t fuck with Philosophy - You have to spend years studying philosophy to even get to the point where you can evaluate whether a philosopher has something useful to say or is just completely full of shit.
BTW, the “G” in GPT simply stands for “generative”, it means that the AI “machine”, or “model”, or “instance”, or whatever we want to call it, “software”, even… learns as it goes, and incorporates that learning to evolve, basically in real time. In the past, an AI machine would be taught in a lab, and if the maker wanted it to learn and grow, it would have to take the machine offline, and feed it new data, check that it was giving the desired results, and then bring back online. with the latest version of this technology, the machine is constantly learning, and evolving, it doesn’t need to be taken offline to learn and grow.
Which leads to the next point, the real point, this technology may be capable of ending humanity, but probably not any time soon. Looking at an app like chatGPT, its horribly innacurate, mistake-prone, full of logical errors. It’s not anything you can even count on to write a reasonably succinct article that hits all the salient points, unless its a really simple topic. It’s easy to test my assertion, simply log in to chatGPT and start feeding it prompts. It becomes quite apparent that the technology is nowhere near ready for primetime, at least not for your average, every day user.
this technology may be capable of ending humanity
I agree, but it’s gonna be because some C-Suite dweeb fires all some nuclear engineer tasked with emergency response and replaces them with a madlibs generator, and then a crisis happens…
And from what I understand, all they “learn” to do is predict what letter goes next. There’s still no cognitive process, no manipulation of symbols, no abstraction of concepts. Nothing that a mind does. It’s still just a fancy weighted random number generator. The resulting strings of text have no semantic value and only resemble speech. We’re interpreting them as having meaning because most people don’t look under the hood. It’s linguistic Pareidolia. Our highly tuned pattern recognition that we rely on to communicate using speech, body language, and so forth is failing in the face of an object that closely resembles speech.
this is a major plot point in (CW: body horror, violence, profound existential dread, mental illness) Blindsight. Spoilers for Blindsight:
spoiler
A linguist spends hours talking with an alien entity before concluding that it has no semantic understanding of it’s speech. It built a model of human speech by intercepting radio communications and was using that model to “communicate”, but it was just mimicking human behavior it had observed. it had no understanding or awareness of why humans were engaging in that behavior or what it did, it just knew that they did it and was mimicking their behavior to get closer to them, the way some predators mimic their prey
Looking at an app like chatGPT, its horribly innacurate, mistake-prone, full of logical errors.
I think we need to push back against claiming that it makes mistakes, logical failures, “hallucinations” or “lies”. It can’t do those things. It’s a computer, and computers do exactly what they’re told. The problem is that the user often doesn’t understand exactly what they’re telling the computer to do. People prompt ChatGPT to answer questions because they think it’s smart, that it understands what they’re typing, that it can think and consider and solve problems. But it doesn’t do any of that. It just compares their prompt to it’s data set and assembles a string of letters that resembles the data set. It’s not making mistakes, it’s doing exactly what it was designed to do; Take an input and produce an output based on the statistical weights of it’s data set. We’re regularly fooling ourself because the output is in letters, not numbers, and we’re attributing meaning to those letters where none exists. If it gets the answer “right” it’s pure luck; There just happened to be enough text strings in it’s training set, and it weighted the values in it’s set in the right way, for a string of output that happens to resemble a correct answer.
This is compounded enormously because, as I understand it, the bazingas who designed these things built a complete black box - They have no way of determining why the LLM generated the outputs that it did. Presumably, unless there is some really weird shit going on, those outputs are deterministic - Given the same inputs the machine should produce the same outputs.
And from what I understand, all they “learn” to do is predict what letter goes next. There’s still no cognitive process, no manipulation of symbols, no abstraction of concepts.
Fascinating but not surprising I guess
LLMs definitely are not the Magic that a lot of idiot techbros think they are, but it’s a mistake to underestimate the technology because it “only generates the next token”. The human brain only generates the next set of neural activations given the previous set of neural activations, and look at how far our intelligence got us.
The capabilities of these things scale with compute used during training, and some of the largest companies on earth are currently in an arms race to throw more and more compute at them. This Will Probably Not End Well. We went from AI barely being able to form a coherent sentence to AI suddenly being a bioterrororism risk in like 2 years because a bunch of chemistry papers were in its training data and now it knows how to synthesize novel chemical warfare agents.
It doesn’t matter whether or not the machine understands what it’s doing when it’s enabling the proliferation of WMDs, or going rogue to achieve some Incoherent goal it extrapolated from it’s training, you’re still Dead at the end.
And from what I understand, all they “learn” to do is predict what letter goes next.
https://thegradient.pub/othello/
It can be difficult to tease out exactly how a neural network is modeling its training data, but claiming that it’s solely predicting the next letter is reductive to the point of being wrong.
That aside, I also just think people are being silly. If an AI can write working code (or beat chess grand masters every time), then obviously something interesting is going on, and protestations that it’s not really thinking and reasoning for realz in a real way, are just kinda obnoxious.
For your consideration:
https://www.lihpao.com/what-cultures-don-t-circumcise/
There’s also a bonus struggle session in the comments section on the fully AI-generated article.
the machine is constantly learning, and evolving, it doesn’t need to be taken offline to learn and grow.
You know I had two thoughts
First, if you want to keep it growing in a direction you consider useful, you need to have humans constantly evaluating it’s outputs. If it’s a black box and we can’t untangle it’s programming the only way to tweak it is to look at what it’s outputting, decide if that’s desirable, and weight the results manually. If it’s not being constantly supervised who knows what it’s going to turn in to. So you’re rate limited by the number of people in the global south you can hire to read it’s outputs.
Second - the people evaluating it’s outputs impose hard limits and biases. If you’ve got the thing spitting out complex maths or chemical formulas the only way to train it is to have someone who understands complex maths or chemical formulas evaluate the outputs. If it gets “too smart” and starts outputting things no one can evaluate you can’t falsify the outputs anymore and you’ve hit an end point. It’s also being trained by people with limited knowledge, lots of biases they don’t know they have, and a propensity to get things wrong. This has already been a problem - NYCs famous black people oppression computer that supposedly predicted crimes when Bloomberg was mayor, and the other case I heard of was some system in the Nordics that was supposed to assess welfare eligibility. The NYC Crimestat computer was a digital Klansman, and the Nordic welfare computer caused all kinds of problems due to biases on behalf of the programmers. Now we’re all excited about AIs that aren’t even programmed, they’re generating their own incomprehensible code that is influenced by the biases of the bazinga techbros training them.
Third - If you don’t know how it’s generating it’s outputs you have no idea what outputs it will generate in the future. Like yeah, you can test it an arbitrarily high number of times and say "Oh it’s correct 99.x% of the time, but as the stakes get higher and the operations become more complex that tricky little x% is going to get more and more problematic. For one - It’s still running on a digital computer, so it’s still deterministic, but we’ve apparently already hit a point where the code is no longer human-interpretable. So you can’t debug it. If it starts doing something undesirable all you can do is boot an earlier back-up and try to train it again. Second, you can’t debug it. When it hits an error or something you have no idea what it will do. that’s fine if it’s running the voice lines for an NPC but a big problem if it’s controlling the RCS on a rocket re-entry. We’re already at the point where high-tech stuff blows up because there are so many lines of spaghetti code that no one knows what will happen when it’s all put to work. Now you’re hooking up complex systems to a black box controller and just hoping that it won’t throw an error or do something unexpected, because testing it is, at best, very difficult.
assess welfare eligibility
Shouldn’t this be dead simple? The law sets the requirements for welfare, the machine looks at your income or whatever and checks if it’s within those requirements.
You know, I had another thought;
With great intelligence comes great insanity.
There’s apparently a pretty strong correlation with doing really well on “intelligence” tests and having a diagnosable mental illness. I’ve heard that really smart people are also more susceptible to certain kinds of delusions because being real good at pattern matching doesn’t mean the patterns you’re noticing are significant, or even really there. But the thinking goes that “smart” people are better at coming up with arguments to support their false beliefs and finding things they think are evidence of their false beliefs, so delusion in “smart” people might be harder to counter than delusion in less “smart” people
(unitary intelligence isn’t real kill the IQ test in your head)
Second - the people evaluating it’s outputs impose hard limits and biases.
this is probably my biggest beef with it. GIGO. garbage in, garbage out I think
Third - If you don’t know how it’s generating it’s outputs you have no idea what outputs it will generate in the future.
Legit point. related to point two also…
it means that the AI “machine”, or “model”, or “instance”, or whatever we want to call it, “software”, even…
I’m not trying to be a pedant when I say this, but I thought that the term generative means that it produces data, in contrast to a discriminative model, which produces discrete (or even fuzzy) classifications from some data. I also thought the term for what you’re describing where the system learns as it does the generation (or inference for discriminators) was “online learning”
I’m not an expert so I will defer to your definition. I am just going with what chatGPT told me. (no, I am not joking…) cheers :)
Fuck, that’s crazy. Thankfully, there are so many people who would rather spend two hours fixing a product like this, than throwing it away.
Team “I’ve taken five years off my life huffing solder fumes but I saved 5$ repairing this lamp!” (No really I am that guy)
Another thing; As a computer, no matter what the Bazingas say, ChatGPT is deterministic. Computers cannot produce true random numbers. It’s mathematically and computationally impossible. With the exact same input the output will be the same, every time, unless a gamma ray flips a bit. Computer programmers cheat by referencing clock time or whatever, but the only way to get true randomness in to a computer program is by getting the input from an external source. People use physical dice rollers, people use the temperature, I’ve seen one that gets randomness from the motion of fish in a fish tank. But you have to go outside.
So whatever ChatGPT is doing in it’s black box, no matter how hard it is to track down or figure out, is deterministic due to the hard constraints of binary computation.
Isn’t the fishtank one a cybersecurity company in SF? I can’t remember the exact one, but it’s in their front office as a display piece.
I also love the idea of pseudo-random numbers, since they’re technically less random, but feel correct
pseudo-random numbers, since they’re technically less random, but feel
They’re also more useful in a lot of cases, because with pseudorandom number generation, you can certify constraints on the distributions that you’re sampling from and guarantee a certain level of stability of the behavior of the algorithm. As a result, there are a few settings where we can get higher entropy pseudorandom numbers than what we can get out of truly random samples of external processes.
Reality is deterministic. If you roll dice the exact same way under the exact same conditions it will result in the same number every time. This is a fundamental principal of physics. Randomness is an illusion of chaos which is defined as extreme sensitivity to initial conditions. AI models are absolutely sensitive to initial conditions and is perfectly capable of being chaotic.
There’s plenty of reasons to criticize AI systems and their uses but saying it’s deterministic is not one of them. It just screams of illiteracy in the topic.
The practical point is to not use uneducated arguments against AI. As I said, there are many reasons to criticize the abilies of AI systems but their ability to be sufficiently random is not one of them. It’s just demonstrably false. You can’t look at the images dall-e outputs and be like “yeah this is too deterministic”.
I remember suggesting a 10 meter Ethernet cable not plugged into anything at the other end to gather background radiation.
That said, I hate this critique for RPG dice. All that’s necessary is for the players not to know the outcome of the next roll. You could have a list of numbers if you wanted.
Spike Jones directed this lmao
Hey y’all remember back in the 90s when Koko the Gorilla was using “sign language” and everyone was super excited about it and thought it was so cool and couldn’t wait to receive some gorilla wisdom and they were all ignoring the linguists and sign language experts yelling that Koko was actually doing what they thought she was?
Please let Kanzi be legit I need this :powercry-2:
I want the silly monkeys to know that I love them.
