First, let me say that what broke me from the herd at lesswrong was specifically the calls for AI pauses. That somehow ‘rationalists’ are so certain advanced AI will kill everyone in the future (pDoom = 100%!) that they need to commit any violent act needed to stop AI from being developed.
The flaw here is that there’s 8 billion people alive right now, and we don’t actually know what the future is. There are ways better AI could help the people living now, possibly saving their lives, and essentially eliezer yudkowsky is saying “fuck em”. This could only be worth it if you actually somehow knew trillions of people were going to exist, had a low future discount rate, and so on. This seems deeply flawed, and seems to be one of the points here.
But I do think advanced AI is possible. And while it may not be a mainstream take yet, it seems like the problems current AI can’t solve, like robotics, continuous learning, module reuse - the things needed to reach a general level of capabilities and for AI to do many but not all human jobs - are near future. I can link deepmind papers with all of these, published in 2022 or 2023.
And if AI can be general and control robots, and since making robots is a task human technicians and other workers can do, this does mean a form of Singularity is possible. Maybe not the breathless utopia by Ray Kurzweil but a fuckton of robots.
So I was wondering what the people here generally think. There are “boomer” forums I know of where they also generally deny AI is possible anytime soon, claim GPT-n is a stochastic parrot, and make fun of tech bros as being hypesters who collect 300k to edit javascript and drive Teslas*.
I also have noticed that the whole rationalist schtick of “what is your probability” seems like asking for “joint probabilities”, aka smoke a joint and give a probability.
Here’s my questions:
-
Before 2030, do you consider it more likely than not that current AI techniques will scale to human level in at least 25% of the domains that humans can do, to average human level.
-
Do you consider it likely, before 2040, those domains will include robotics
-
If AI systems can control robotics, do you believe a form of Singularity will happen. This means hard exponential growth of the number of robots, scaling past all industry on earth today by at least 1 order of magnitude, and off planet mining soon to follow. It does not necessarily mean anything else.
-
Do you think that mass transition where most human jobs we have now will become replaced by AI systems before 2040 will happen
-
Is AI system design an issue. I hate to say “alignment”, because I think that’s hopeless wankery by non software engineers, but given these will be robotic controlling advanced decision-making systems, will it require lots of methodical engineering by skilled engineers, with serious negative consequences when the work is sloppy?
*“epistemic status”: I uh do work for a tech company, my job title is machine learning engineer, my girlfriend is much younger than me and sometimes fucks other dudes, and we have 2 Teslas…
There are “boomer” forums I know of where they also generally deny AI is possible anytime soon, claim GPT-n is a stochastic parrot, and make fun of tech bros as being hypesters who collect 300k to edit javascript and drive Teslas*.
i hate rationalists too but this is literally a correct take
People when the sound like a person machine sounds like a person
idly:
The Gfycat service is being discontinued. Please save or delete your Gfycat content by visiting https://www.gfycat.com and logging in to your account. After September 1, 2023, all Gfycat content and data will be deleted from gfycat.com
but I chuckled nonetheless
Nerd religion.
At the old SneerClub, /r/SneerClub on reddit, which is still up but no longer active, we had two policies relevant to this situation: (1) serious posts are marked “NSFW”, so that people don’t accidentally click through to the rambling mind palace tour of a random interloper without some idea of what they were in for; (2) if you use language like “epistemic status” as if everybody knows what that means, which is to say as if you had blithely assumed everybody was in your discursive club, I ban you.
A third, as well, was for regular users to please not encourage the interlopers in this wilfully (acquiredly?) solipsistic behaviour.
I have to ask, on the matter of (2): why? Everybody who knows what the term “epistemic status” means and what it socially signifies also knows that it’s entirely peculiar to this group “rationalists”, a group of people which the forum here exists to target, and negatively. I ask because it says something about the rest of the post.
The opening wants to put you in a distal relationship both with the rationalists and with any among the criticisms whomof which are not criticisms of AI doom shithousery. That would put you in a relatively small club, being people who are broadly aligned with everything else that the rationalists have to say, but who specifically do not think that Skynet is coming, and who think that this is the one area where rationalists get it wrong. At the same time, however, this term “epistemic status” functions to signify that you’re fairly sure you’re among friends on this forum, while at the same time what you want to do is enquire whether that is the case: you want to find out if people on this forum share that singular bad feeling with you.
This is rather a mess of contrasting significations: usually this happens when somebody is speaking rather out of both sides of their mouth.
What’s being signified when you point to “boomer forums”? That’s an “among friends” usage: you’re free to denigrate the boomer fora here. And then once again you don’t know yet if this is one of those “boomer forums”, or you wouldn’t have to ask.
What people in their droves are now desperate to ask, I will ask too: which is it dummy? Take the stopper out of your speech hole and tell us how you really feel.
A bad habit rationalism teaches is to treat a stock verbiage of polite and open discussion as on the one hand (a) integral to productive conversation, and (b) automatically generative of productive conversation. But people aren’t like that, because people are in general really smart listeners (and readers) when it comes to figuring out what is stock verbiage and what is meant in earnest. This doesn’t mean that they always draw the right conclusion if they come right out and say “I think this is somebody honest trying too hard” or “this person is full of shit” - but that initial intuition is usually right on target, and it has to be unlearned with a great deal of training in the tenets of the rationalist cult in order to guarantee the cult’s intellectually pointless hierarchies of “good discussion”, which are nonetheless crucial as a tool of enforcement within the cult’s social order.
Anyway, the question in the above paragraph is open.
tbh I read the statement about epistemic status as ironic. I was disabused of this notion rapidly.
A bad habit rationalism teaches is to treat a stock verbiage of polite and open discussion as on the one hand (a) integral to productive conversation, and (b) automatically generative of productive conversation. But people aren’t like that, because people are in general really smart listeners (and readers) when it comes to figuring out what is stock verbiage and what is meant in earnest.
Thanks for articulating this.
we held off for a bit cos we don’t want to be actively unkind to the recovering rationalists, and he was our first ardent debate bro actually on the instance, but he rapidly also became our first 24 hour ban of a local account rather than a federated one. perhaps his posting will improve a day hence!
I appreciated this post because it never occurred to me that the “thumb might be on the scales” for the “rules for discourse” that seems to be the norm around the rat forms. I personally ignore most of it, however, the “ES” rat phrase is simply saying, “I know we humans are biased observers, this is where I’m coming from”. If the topic were renewable energy and I was the ‘head of extraction at BP’, you can expect that whatever I have to say is probably biased against renewable energy.
My other thought reading this was : what about the truth. Maybe the mainstream is correct about everything. “Sneer club” seems to be mostly mainstream opinions. That’s fine I guess but the mainstream is sometimes wrong about issues that have been poorly examined or near future events. The collective opinions of everyone don’t really price in things that are about to happen, even if it’s obvious to experts. For example, the mainstream opinion on covid was usually lagging several weeks behind Zvi’s posts on lesswrong.
Where I am going with this is you can point out bad arguments on my part, but I mean in the end, does truth matter? Like are we here to score points on each other or share what we think reality is or will in the very near future be?
I would hardly consider myself in favour of “the mainstream”, but I also know that what counts as “mainstream” is irreducibly dependent on your point of view. As far as I’m concerned a great deal of anti-“mainstream” opinion is reactionary and/or stupid, so anti-“mainstream” only by default. A stopped clock, famously, tells the truth twice a day - whether its on CBS or LessWrong. If you want the “truth” I recommend narrowing your focus until you start making meaningful distinctions. I hope that as comfortably vitiates your point as it should.
Next time it would be polite to answer the fucking question.
Next time it would be polite to answer the fucking question.
Sorry sir:
*I have to ask, on the matter of (2): why? * I think I answered this.
What’s being signified when you point to “boomer forums”? That’s an “among friends” usage: you’re free to denigrate the boomer fora here. And > then once again you don’t know yet if this is one of those “boomer forums”, or you wouldn’t have to ask.
What people in their droves are now desperate to ask, I will ask too: which is it dummy? Take the stopper out of your speech hole and tell us how > you really feel.
I am not sure what you are asking here, sir. It’s well known to those in the AI industry that a profound change is upon us and that GPT-4 shows generality for it’s domain, and robotics generality is likely also possible using a variant technique. So individuals unaware of this tend to be retired people who have no survival need to learn any new skills, like my boomer relatives. I apologize for using an ageist slur.
Epistemic Status: Single/Cali girl ;)
Maybe the mainstream is correct about everything. “Sneer club” seems to be mostly mainstream opinions.
Lurk moar.
For example, the mainstream opinion on covid was usually lagging several weeks behind Zvi’s posts on lesswrong.
Heaven forbid the mainstream take a few weeks to figure shit out when presented with new information instead of violently changing gears every time a new story or rumor gets published.
For anyone curious: https://www.lesswrong.com/s/rencyawwfr4rfwt5C
My favorite quotes from within:
Going on walks considered fine for some reason, very strange.
My current best thought for how to do experiments quickly is medical cruise ships in international waters. […] Medical cruise ships are already an established way to do things without running into regulatory problems.
We are willing to do things that people find instinctively repugnant, provided they save lives while at least not hurting the economy. How could we accomplish this?
ooooookay longpost time
first off: eh wtf, why is this on sneerclub? kinda awks. but I’ll try give it a fair and honest answer.
First, let me say that what broke me from the herd at lesswrong was specifically the calls for AI pauses.
look, congrats on breaking out, but uh… you’re still wearing the prison jumpsuit in the grocery store and that’s why people are looking at you weirdly
“yay you got out” but you got only half the reason right
take some time and read this
This seems deeply flawed
correct
But I do think advanced AI is possible
one note here: “plausible” vs “possible” are very divergent paths and likelihoods
in the Total Possible Space Of All Things That Might Ever Happen, of course it’s possible, but so are many, many other things
it seems like the problems current AI can’t solve, like robotics, continuous learning, module reuse - the things needed to reach a general level of capabilities and for AI to do many but not all human jobs - are near future
eh. this ties back to my opener - you’re still too convinced about something on essentially no grounded basis other than industry hype-optimism
I can link deepmind papers with all of these, published in 2022 or 2023.
look I don’t want to shock you but that’s basically what they get paid to do. and (perverse) incentives apply - of course goog isn’t just going to spend a couple decabillion then go “oh shit, hmm, we’ve reached the limits of what this can do. okay everyone, pack it in, we’re done with this one!”, they’re gonna keep trying to milk it to make some of those decabillions back. and there’s plenty of useful suckers out there
And if AI can be general and control robots, and since making robots is a task human technicians and other workers can do, this does mean a form of Singularity is possible. Maybe not the breathless utopia by Ray Kurzweil but a fuckton of robots.
okay this is a weird leap and it’s borderline LW shittery so I’m not going to spend much effort on it, but I’ll give you this
it doesn’t fucking matter.
even if we do somehow crack even the smallest bit of computational sentience, the plausibility of rapid acting self-reinforcing runaway self-improvement on such a thing is basically nil. we’re 3 years down the line on the Evergreen getting stuck in the suez and fabs shutting down (with downstream orders being cancelled) and as a result of it a number of chips are still effectively unobtanium (even if and when you have piles and piles of money to throw at the problem). multiple industries, worldwide, are all throwing fucking tons of money at the problem to try recover from the slightest little interruption in supply (and like, “slight”, it wasn’t even like fabs burned down or something, they just stopped shipping for a while)
just think of the utter scope of doing robotics. first you have to solve a whole bunch of design shit (which by itself involves a lot of from-principles directed innovation and inspiration and shit). then you have to figure out how to build the thing in a lab. then you have to scale it? which involves ordering thousounds of parts and SKUs from hundred of vendors. then find somewhere/somehow to assemble it? and firmware and iteration and all that shit?
this isn’t fucking age of ultron, and tony’s parking-space fab isn’t a real thing.
this outcome just isn’t fucking likely on any nearby horizon imo
So I was wondering what the people here generally think
we generally think the people who believe this are unintentional suckers or wilful grifters. idk what else to tell you? thought that was pretty clear
There are “boomer” forums I know of where they also generally deny AI is possible anytime soon, claim GPT-n is a stochastic parrot, and make fun of tech bros as being hypesters who collect 300k to edit javascript and drive Teslas*.
wat
I also have noticed that the whole rationalist schtick of “what is your probability” seems like asking for “joint probabilities”, aka smoke a joint and give a probability.
okay this gave me a momentary chuckle, and made me remember JRPhttp://darklab.org/jrp.txt (which is a fun little shitpost to know about)
from here, answering your questions as you asked them in order (and adding just my own detail in areas where others may not already have covered something)
-
no, not a fuck, not even slightly. definitely not with the current set of bozos at the helm or techniques as the foundation or path to it.
-
no, see above
-
who gives a shit? but seriously, no, see above. even if it did, perverse incentives and economic pressures from sweeping hand motion all this other shit stands a very strong chance to completely fuck it all up 60 ways to sunday
-
snore
-
if any of this happens at some point at all, the first few generations of it will probably look the same as all other technology ever - a force-multiplier with humans in the loop, doing things and making shit. and whatever happens in that phase will set the one on whatever follows so I’m not even going to try predict that
*“epistemic status”: I uh do work for a tech company, my job title is machine learning engineer, my girlfriend is much younger than me and sometimes fucks other dudes, and we have 2 Teslas…
…okay? congrats? is that fulfilling for you? does it make you happy?
not really sure why you mentioned the gf thing at all? there’s no social points to be won here
closing thoughts: really weird post yo. like, “5 yud-steered squirrels in a trenchcoat” weird.
look I don’t want to shock you but that’s basically what they get paid to do. and (perverse) incentives apply - of course goog isn’t just going to spend a couple decabillion then go “oh shit, hmm, we’ve reached the limits of what this can do. okay everyone, pack it in, we’re done with this one!”, they’re gonna keep trying to milk it to make some of those decabillions back. and there’s plenty of useful suckers out there
a lot of corporations involved with AI are doing their damndest to damage our relationship with the scientific process by releasing as much fluff disguised as research as they can manage, and I really feel like it’s a trick they learned from watching cryptocurrency projects release an interminable amount of whitepapers (which, itself, damaged our relationship with and expectations from the engineering process)
As someone who went from high school directly into a publishing company as a “web designer” in 1998 I spent the next 20 years assuming that academic work was completely uninfluenced by commercial interests. HCI was academic, UX was commercial. Wasn’t till around 2019 that I started reading ACM papers about HCI from the 70s up. Fuck me was I surprised with how mixed up it all is. ACM interactions magazine published monthly case studies for Apple or did profiles on Jef Raskin talking about HCI for brand loyalty.
Anyway. Point is a published paper doesn’t mean shit if you just read a few because an article pointed you to them. I don’t know. This thread sucks
Preach, as someone inside academia, the bullcrap is real. I very rarely read a paper that hasn’t got a major stats issue—an academic paper is only worth something if you understand it enough to know how wrong it is or there’s plenty of replication/related work building on it, ideally both. (And it’s a technical field with an objective measure of truth but don’t let my colleagues in humanities hear me say that—its not that their work is worthless, its just its not reliable.)
“shitcoiners or oil companies… who wore it best?”
but the rest of your reply reminds me that someone (I think steve or blake?) mentioned a thing here recently about a book on blaming guthenberg for this state of fucking everything up. I want to go read that, and I really need to get around to writing my rantpost about the “the problem of information transfer at scale is that scale is lossy, and this is why … [handwaves at many problems, examples continue]” thing that at least 8 friends of mine have had to put up with in DM over the last few years
They also hyped autonomous cars and the Internet itself including streaming video for years before it was practical. Your filter of “it’s all hype” only works 99 percent of the time.
take some time and read this
I read it. I appreciated the point that human perception of current AI performance can scam us, though this is nothing new. People were fooled by Eliza.
It’s a weak argument though. For causing an AI singularity, functional intelligence is the relevant parameter. Functional intelligence just means “if the machine is given a task, what is the probability it completes the task successfully”. Theoretically an infinite chinese room can have functional intelligence (the machine just looks up the sequence of steps for any given task).
People have benchmarked GPT-4 and it’s got general functional intelligence at tasks that can be done on a computer. You can also just go pay up $20 a month and try it. It’s below human level overall I think, but still surprisingly strong given it’s emergent behavior from computing tokens.
Just I think to summarize your beliefs: rationalists are wrong about a lot of things and assholes. And also the singularity (which predates yuds existence) is not in fact possible by the mechanism I outlined.
I think this is a big crux here. It’s one thing if its a cult around a false belief. It’s kind of a problem to sneer at a cult if the core S of it happens to be a true law of nature.
Or an analogy. I think gpt-4 is like the data from the Chicago pile. That data was enough to convince the domain experts then a nuke was going to work to the point they didn’t test Fat Man, you believe not. Clearly machine generality is possible, clearly it can solve every problem you named including, with the help of humans, ordering every part off digikey and loading the pick and place and inspecting the boards and building the wire harnesses and so on.
Just I think to summarize your beliefs
don’t be puttin’ words in my mouth yo
rationalists
this is a big set of very many people and lots of details
are wrong about a lot of things
many of them about many things, yes
and assholes
some, provably
And also the singularity (which predates yuds existence) is not in fact possible by the mechanism I outlined
whether it’s the wet dream of kurzweil or yud or whoever else, doesn’t matter? but as to the details… you’re engaging with this like the rats do (yes, told you, you only half escaped). you “set the example”, and then “test the details”
just … don’t?
the siren song of this is “okay what if I change the details of the experiment slightly?”
we’ve had the trolley problem for ages, doesn’t mean it’s just “solved”. you won’t manage to “solve” whether the singularity can happen or not here, for the same reason
Or an analogy. I think gpt-4 is like the data from the Chicago pile. That data was enough to convince the domain experts then a nuke was going to work to the point they didn’t test Fat Man, you believe not.
Are you mixing up Fat Man and Little Boy? Because Fat Man was an implosion-type bomb, just like the Trinity device. Little Boy was a gun-type. From vague memories of Rhode’s book, they wanted implosion types to maximize Pu weight to kiloton ratio, but it was much less straightforward than a gun-type bomb.
Whaaaaat? I don’t come here for this shit
I don’t really see much likelihood in a singularity though, there’s probably a bunch of useful shit you could work out if you analysed the right extant data in the right way but there’s huge amounts of garbage data that it’s not obvious is garbage.
My experience in research indicates to me that figuring shit out is hard and time consuming, and “intelligence” whatever that is has a lot less to do with it than having enough resources and luck. I’m not sure why some super smart digital mind would be able to do science much faster than humans.
Physics is a bitch and there are just sort of limits on how awesome technology can be. Maybe I’m wrong but it seems like digital intelligence would be more useful for stuff like finding new antibiotics than making flying nanomagic fabricator paperclip drones.
My experience in research indicates to me that figuring shit out is hard and time consuming, and “intelligence” whatever that is has a lot less to do with it than having enough resources and luck. I’m not sure why some super smart digital mind would be able to do science much faster than humans.
That’s right. Eliezer’s LSD vision of the future where a smart enough AI just figures it all out with no new data is false.
However, you could…build a fuckton of robots. Have those robots do experiments for you. You decide on the experiments, probably using a procedural formula. For example you might try a million variations of wing design, or a million molecules that bind to a target protein, and so on. Humans already do this actually in those domains, this is just extending it.
For example you might try […] a million molecules that bind to a target protein
well not millions but tens of thousands, yes we have that, it’s called high throughput screening. it’s been around for some time
have you noticed some kind of medical singularity? is every human disease curable by now? i don’t fucking think so
that’s because you’re automating glorified liquid transfer from eppendorf A to eppendorf B, followed by simple measurement like fluorescence. you still have to 1. actually make all of this shit and make sure it’s pure and what you ordered, then 2. you have to design an experiment that will tell you something that you measure, and be able to interpret it correctly, then 3. you need to be sure that you’re doing the right thing in the first place, like not targeting the wrong protein (more likely than you think), and then 4. when you have some partial result, you latch to it and improve it piece by piece, making sure that it will actually get where it needs to, won’t shred patient’s liver instantly and so on (more likely than you think)
while 1 is at initial stages usually subcontracted to poor sods at entity like enamine ltd, 1, 4 are infinite career opportunities for medicinal/organic chemists and 2, 3 for molecular biologists, because all AI attempts at any of that that i’ve seen were spectacular failures and the only people that were satisfied with it were people who made these systems and published a paper about them. especially 4 is heavily susceptible to garbage in garbage out situations, and putting AI there only makes matters worse
is HTS a good thing? if you can afford it, it relieves you from the most mind numbing task out there. if you can’t you still do all of this by hand. (it seems to me that it escapes you that all of this shit costs money) is this a new thing? also no. since 90s you can buy automated flash chromatographic column, it’s a box where you put dirty compound in one tube and get purified compound in other tubes. guess what took me entire yesterday? yes, it’s flash columns by hand because my uni doesn’t have a budget for that. would my paper come up faster if i had a combiflash? maybe, would it be any better if i had 5? no, because all the hard bits aren’t automated away, shit breaks all the time, things work different than you think and sometimes it’s that what makes it noticeable, and so on and so on
You know, I thought that moving sneerclub onto lemmy meant we probably would not get that familiar mix of rationalists, heterodox rationalists, and just-left-but-still-mired-in-the-mindset ex-rationalists that swing by and want to quiz sneerclub. Maybe we’re just that irresistible.
from 2011-2013 i was getting these guys email me directly about roko’s basilisk because lesswrong had banned discussion and rationalwiki was the only place even mentioning it
now they work hard to seek us out even here
i hope the esteemed gentleposter realises that there are no recoverable good parts and it’s dumbassery all the way down sooner rather than later, preferably before posting again
Maybe we could make an explicit sub-lemmy for indulging in maladaptive debating. It’s my guilty pleasure.
Jesus fuck. Idk about no good parts, the bits that are unoriginal are sometimes interesting (e.g. distance between model and reality, metacognition is useful sometimes etc) it would just be more useful if they like produced reading lists instead of pretending to be smort
No good original parts. Yudkowsky can write and explain fine when he’s not talking about his dumb nonsense.
Hi David. Reason I dropped by was the whole concept of knowing the distant future with too much certainty seemed like a deep flaw, and I have noticed lesswrong itself is full of nothing but ‘cultist’ AI doomers. Everyone kinda parrots a narrow range of conclusions, mainly on the imminent AGI killing everyone, and this, ironically, doesn’t seem very rational…
I actually work on the architecture for current production AI systems and whenever I mention approaches that do work fine and suggest we could control more powerful AI this way, I get downvoted. So I was trying to differentiate between:
A. This is a club of smart people, even smarter than lesswrongers who can’t see the flaws!
B. This is a club of well, the reason I called it boomers was I felt that the current news and AI papers make each of the questions I asked a reasonable and conservative outcome. For example posters here are saying for (1), “no it won’t do 25% of the jobs”. That was not the question, it was 25% of the tasks. Since for example Copilot already writes about 25% of my code, and GPT-4 helps me with emails to my boss, from my perspective this is reasonable. The rest of the questions build on (1).
I actually work on the architecture for current production AI systems and whenever I mention approaches that do work fine and suggest we could control more powerful AI this way, I get downvoted.
LW isn’t looking for technical practical solutions. They want plausible sci-fi that fits their narrative. Actually solving the problems they worry about would mean there’s no reason for the cult to exist, so why would they upvote that?
Overall LW seems to be dead wrong about predicting modern AI systems. They anticipated that there was this general intelligence quality that would enable problem solving, escape, instrumental convergence, etc. However what ended up working was approximating functions really hard. The existence of ChatGPT without a singularity is a crisis for LW. No longer can they safely pontificate and write Harry Potter/The Culture fanfiction; now they must confront the practical reality of the monsters under their bed looking an awful lot more like dust bunnies.