First, let me say that what broke me from the herd at lesswrong was specifically the calls for AI pauses. That somehow ‘rationalists’ are so certain advanced AI will kill everyone in the future (pDoom = 100%!) that they need to commit any violent act needed to stop AI from being developed.
The flaw here is that there’s 8 billion people alive right now, and we don’t actually know what the future is. There are ways better AI could help the people living now, possibly saving their lives, and essentially eliezer yudkowsky is saying “fuck em”. This could only be worth it if you actually somehow knew trillions of people were going to exist, had a low future discount rate, and so on. This seems deeply flawed, and seems to be one of the points here.
But I do think advanced AI is possible. And while it may not be a mainstream take yet, it seems like the problems current AI can’t solve, like robotics, continuous learning, module reuse - the things needed to reach a general level of capabilities and for AI to do many but not all human jobs - are near future. I can link deepmind papers with all of these, published in 2022 or 2023.
And if AI can be general and control robots, and since making robots is a task human technicians and other workers can do, this does mean a form of Singularity is possible. Maybe not the breathless utopia by Ray Kurzweil but a fuckton of robots.
So I was wondering what the people here generally think. There are “boomer” forums I know of where they also generally deny AI is possible anytime soon, claim GPT-n is a stochastic parrot, and make fun of tech bros as being hypesters who collect 300k to edit javascript and drive Teslas*.
I also have noticed that the whole rationalist schtick of “what is your probability” seems like asking for “joint probabilities”, aka smoke a joint and give a probability.
Here’s my questions:
-
Before 2030, do you consider it more likely than not that current AI techniques will scale to human level in at least 25% of the domains that humans can do, to average human level.
-
Do you consider it likely, before 2040, those domains will include robotics
-
If AI systems can control robotics, do you believe a form of Singularity will happen. This means hard exponential growth of the number of robots, scaling past all industry on earth today by at least 1 order of magnitude, and off planet mining soon to follow. It does not necessarily mean anything else.
-
Do you think that mass transition where most human jobs we have now will become replaced by AI systems before 2040 will happen
-
Is AI system design an issue. I hate to say “alignment”, because I think that’s hopeless wankery by non software engineers, but given these will be robotic controlling advanced decision-making systems, will it require lots of methodical engineering by skilled engineers, with serious negative consequences when the work is sloppy?
*“epistemic status”: I uh do work for a tech company, my job title is machine learning engineer, my girlfriend is much younger than me and sometimes fucks other dudes, and we have 2 Teslas…
Whaaaaat? I don’t come here for this shit
I don’t really see much likelihood in a singularity though, there’s probably a bunch of useful shit you could work out if you analysed the right extant data in the right way but there’s huge amounts of garbage data that it’s not obvious is garbage.
My experience in research indicates to me that figuring shit out is hard and time consuming, and “intelligence” whatever that is has a lot less to do with it than having enough resources and luck. I’m not sure why some super smart digital mind would be able to do science much faster than humans.
Physics is a bitch and there are just sort of limits on how awesome technology can be. Maybe I’m wrong but it seems like digital intelligence would be more useful for stuff like finding new antibiotics than making flying nanomagic fabricator paperclip drones.
My experience in research indicates to me that figuring shit out is hard and time consuming, and “intelligence” whatever that is has a lot less to do with it than having enough resources and luck. I’m not sure why some super smart digital mind would be able to do science much faster than humans.
That’s right. Eliezer’s LSD vision of the future where a smart enough AI just figures it all out with no new data is false.
However, you could…build a fuckton of robots. Have those robots do experiments for you. You decide on the experiments, probably using a procedural formula. For example you might try a million variations of wing design, or a million molecules that bind to a target protein, and so on. Humans already do this actually in those domains, this is just extending it.
For example you might try […] a million molecules that bind to a target protein
well not millions but tens of thousands, yes we have that, it’s called high throughput screening. it’s been around for some time
have you noticed some kind of medical singularity? is every human disease curable by now? i don’t fucking think so
that’s because you’re automating glorified liquid transfer from eppendorf A to eppendorf B, followed by simple measurement like fluorescence. you still have to 1. actually make all of this shit and make sure it’s pure and what you ordered, then 2. you have to design an experiment that will tell you something that you measure, and be able to interpret it correctly, then 3. you need to be sure that you’re doing the right thing in the first place, like not targeting the wrong protein (more likely than you think), and then 4. when you have some partial result, you latch to it and improve it piece by piece, making sure that it will actually get where it needs to, won’t shred patient’s liver instantly and so on (more likely than you think)
while 1 is at initial stages usually subcontracted to poor sods at entity like enamine ltd, 1, 4 are infinite career opportunities for medicinal/organic chemists and 2, 3 for molecular biologists, because all AI attempts at any of that that i’ve seen were spectacular failures and the only people that were satisfied with it were people who made these systems and published a paper about them. especially 4 is heavily susceptible to garbage in garbage out situations, and putting AI there only makes matters worse
is HTS a good thing? if you can afford it, it relieves you from the most mind numbing task out there. if you can’t you still do all of this by hand. (it seems to me that it escapes you that all of this shit costs money) is this a new thing? also no. since 90s you can buy automated flash chromatographic column, it’s a box where you put dirty compound in one tube and get purified compound in other tubes. guess what took me entire yesterday? yes, it’s flash columns by hand because my uni doesn’t have a budget for that. would my paper come up faster if i had a combiflash? maybe, would it be any better if i had 5? no, because all the hard bits aren’t automated away, shit breaks all the time, things work different than you think and sometimes it’s that what makes it noticeable, and so on and so on
from 2011-2013 i was getting these guys email me directly about roko’s basilisk because lesswrong had banned discussion and rationalwiki was the only place even mentioning it
now they work hard to seek us out even here
i hope the esteemed gentleposter realises that there are no recoverable good parts and it’s dumbassery all the way down sooner rather than later, preferably before posting again
Maybe we could make an explicit sub-lemmy for indulging in maladaptive debating. It’s my guilty pleasure.
Hi David. Reason I dropped by was the whole concept of knowing the distant future with too much certainty seemed like a deep flaw, and I have noticed lesswrong itself is full of nothing but ‘cultist’ AI doomers. Everyone kinda parrots a narrow range of conclusions, mainly on the imminent AGI killing everyone, and this, ironically, doesn’t seem very rational…
I actually work on the architecture for current production AI systems and whenever I mention approaches that do work fine and suggest we could control more powerful AI this way, I get downvoted. So I was trying to differentiate between:
A. This is a club of smart people, even smarter than lesswrongers who can’t see the flaws!
B. This is a club of well, the reason I called it boomers was I felt that the current news and AI papers make each of the questions I asked a reasonable and conservative outcome. For example posters here are saying for (1), “no it won’t do 25% of the jobs”. That was not the question, it was 25% of the tasks. Since for example Copilot already writes about 25% of my code, and GPT-4 helps me with emails to my boss, from my perspective this is reasonable. The rest of the questions build on (1).
I actually work on the architecture for current production AI systems and whenever I mention approaches that do work fine and suggest we could control more powerful AI this way, I get downvoted.
LW isn’t looking for technical practical solutions. They want plausible sci-fi that fits their narrative. Actually solving the problems they worry about would mean there’s no reason for the cult to exist, so why would they upvote that?
Overall LW seems to be dead wrong about predicting modern AI systems. They anticipated that there was this general intelligence quality that would enable problem solving, escape, instrumental convergence, etc. However what ended up working was approximating functions really hard. The existence of ChatGPT without a singularity is a crisis for LW. No longer can they safely pontificate and write Harry Potter/The Culture fanfiction; now they must confront the practical reality of the monsters under their bed looking an awful lot more like dust bunnies.
Jesus fuck. Idk about no good parts, the bits that are unoriginal are sometimes interesting (e.g. distance between model and reality, metacognition is useful sometimes etc) it would just be more useful if they like produced reading lists instead of pretending to be smort
No good original parts. Yudkowsky can write and explain fine when he’s not talking about his dumb nonsense.
You know, I thought that moving sneerclub onto lemmy meant we probably would not get that familiar mix of rationalists, heterodox rationalists, and just-left-but-still-mired-in-the-mindset ex-rationalists that swing by and want to quiz sneerclub. Maybe we’re just that irresistible.
Content Warning: Ratspeak
spoiler
Let’s say that tomorrow, they build AGI on HP/Cray Frontier. It’s human equivalent. Mr Frontier is rampant or whatever and wants to improve himself. In order to improve himself he will need to create better chips. He will need approximately 73 thousand copies of himself just to match the staff of TSMC, but there’s only one Frontier. And that’s to say nothing of the specialized knowledge and equipment required to build a modern fab, or the difficulty of keeping 73 thousand copies of himself loyal to his cause. That’s just to make a marginal improvement on himself, and assuming everyone is totally ok with letting the rampant AI get whatever it wants. And that’s just the ‘make itself smarter’ part, which everything else is contingent on; it assumes that we’ve solved Moravec’s paradox and all of the attendant issues of building robots capable of operating at the extremes of human adaptability, which we have not. Oh and it’s only making itself smarter at the same pace TSMC already was.
The practicalities of improving technology are generally skated over by aingularatians in favor of imagining technology as a magic number that you can just throw “intelligence” at to make it go up.
What I’m trying to get at is that the practicalities of improving technology are generally skated over by aingularatians in favor of imagining technology as a magic number that you can just throw “intelligence” at to make it go up.
this is where the singularity always lost me. like, imagine, you build an AI and it maxes out the compute in its server farm (a known and extremely easy to calculate quantity) so it decides to spread onto the internet where it’ll have infinite compute! well congrats, now the AI is extremely slow cause the actual internet isn’t magic, it’s a network where latency and reliability are gigantic issues, and there isn’t really any way for an AI to work around that. so singulatarians just handwave it away
or like when they reach for nanomachines as a “scientific” reason why the AI would be able to exert godlike influence on the real world. but nanomachines don’t work like that at all, it’s just a lazy soft sci-fi idea that gets taken way too seriously by folks who are mediocre at best at understanding science
Serious answer not from yudnowsky: the AI doesn’t do any of that. It helps people cheat on their homework, write their code and form letters faster, and brings in revenue. AI owner uses the revenue and buys gpus. With the GPUs they make the AI better. Now it can do a bit more than before and then they buy more GPUs and theoretically this continues until the list of tasks the AI can do includes “most of the labor in a chip fab” and GPUs become cheap and then things start to get crazy.
Same elementary school logic but I mean this is how a nuke works.
wait, so the AI is just your fears about capitalism?
Same elementary school logic but I mean this is how a nuke works.
what. no it isn’t
your imaginary nukes explode forever. in reality, nuke stops exploding when it either runs out of plutonium or is dispersed too much. the energy of nuke is not infinite, it’s large, but most importantly it’s all contained in the device from the beginning
your example also fails at the step of “getting more money forever”, when VC funding runs out or gets dispersed too much entire charade grinds to halt (because SV startups are shielded from commercial failure by that VC money). that state is sometimes called “AI winter”
but nanomachines don’t work like that at all, it’s just a lazy soft sci-fi idea that gets taken way too seriously by folks who are mediocre at best at understanding science
Let’s call this Crichtonitis.
I agree completely. This is exactly where I break with Eliezer’s model. Yes obviously an AI system that can self improve can only do so until it’s either (1) the best algorithm that can run on the server farm (2) finding a better algorithm takes more compute than is worth the investment in current compute
That’s not a god. You do this in an AI experiment now and it might crap out at double or less the starting performance and not even be above the SOTA.
But if robots can build robots, and the current AI progress shows a way to do it (foundation model on human tool manipulation), then…
Genuinely asking, I don’t think it’s “religion” to suggest that a huge speedup in global GDP would be a dramatic event.
(To be read in the voice of an elementary schooler who is a sore loser at make believe): Nuh-uh! My AGI has quantum computers, so it doesn’t get slow from the internet, and, and, and, it builds robots, with jetpacks, and those robots have tiny robots that can go in your brain and and and make your brain explode, and if you say anything mean about me or the AGI it’ll take your brain and clone it and put wires in it and make you think youre getting like, wedgied and stuff, but really youre not but you think you are because it’s really good at making you think it
oh god, rationalists really were those kids and they never grew out of it
Indeed, if distributed computing worked as well as singulatarians fear everyone would be using Beowulf clusters for their workloads instead of AWS.
I will answer these sincerely in as much detail as necessary. I will only do this once, lest my status amongst the sneerclub fall.
- I don’t think this question is well-defined. It implies that we can qualify all the relevant domains and quantify average human performance in those domains.
- See above.
- I think “AI systems” already control “robotics”. Technically, I would count kids writing code for a simple motorised robot to satisfy this. Everywhere up the ladder, this is already technically true. I imagine you’re trying to ask about AI-controlled robotics research, development and manufacturing. Something like what you’d see in the Terminator franchise- Skynet takes over, develops more advanced robotic weapons, etc. If we had Skynet? Sure, Skynet formulated in the films would produce that future. But that would require us to be living in that movie universe.
- This is a much more well-defined question. I don’t have a belief that would point me towards a number or probability, so no answer as to “most.” There are a lot of factors at play here. Still, in general, as long as human labour can be replaced by robotics, someone will, at the very least, perform economic calculations to determine if that replacement should be done. The more significant concern here for me is that in the future, as it is today, people will still only be seen as assets at the societal level, and those without jobs will be left by the wayside and told it is their fault that they cannot fend for themselves.
- Yes, and we already see that as an issue today. Love it or hate it, the partisan news framework produces some consideration of the problems that pop up in AI development.
Time for some sincerity mixed with sneer:
I think the disconnect that I have with the AGI cult comes down to their certainty on whether or not we will get AGI and, more generally, the unearned confidence about arbitrary scientific/technological/societal progress being made in the future. Specifically with AI => AGI, there isn’t a roadmap to get there. We don’t even have a good idea of where “there” is. The only thing the AGI cult has to “convince” people that it is coming is a gish-gallop of specious arguments, or as they might put it, “Bayesian reasoning.” As we say, AGI is a boogeyman, and its primary use is bullying people into a cult for MIRI donations.
Pure sneer (to be read in a mean, high-school bully tone):
Look, buddy, just because Copilot can write spaghetti less tangled than you doesn’t mean you can extrapolate that to AGI exploring the stars. Oh, so you use ChatGPT to talk to your “boss,” who is probably also using ChatGPT to speak to you? And that convinces you that robots will replace a significant portion of jobs? Well, that at least convinces me that a robot will replace you.
Well, that at least convinces me that a robot will replace you.
i am sincerely convinced that VCs who fearmonger about AI are worried that GPT-3 would already do twitter better than them
Oh, they’re quaking in those VC boots. That’s what’ll set off the Big One.
the singularity will be caused by hooking GPT-4 to an account at Silicon Valley Bank
1, 2 : since you claim you can’t measure this even as a thought experiment, there’s nothing to discuss 3. I meant complex robotic systems able to mine minerals, truck the minerals to processing plants, maintain and operate the processing plants, load the next set of trucks, the trucks go to part assembly plants, inside the plant robots unload the trucks and feed the materials into CNC machines and mill the parts and robots inspect the output and pack it and more trucks…culminating in robots assembling new robots.
It is totally fine if some human labor hours are still required, this cheapens the cost of robots by a lot.
- This is deeply coupled to (3). If you have cheap robots, if an AI system can control a robot well enough to do the task as well as a human, obviously it’s cheaper to have robots do the task than a human in most situations.
Regarding (3) : the specific mechanism would be AI that works like this:
Millions of hours of video of human workers doing tasks in the above domain + all video accessible to the AI company -> tokenized compressed description of the human actions -> llm like model. The llm like model thus is predicting “what would a human do”. You then need a model to transform the what to robotic hardware that is built differently than humans, and this is called the “foundation model”: you use reinforcement learning where actual or simulated robots let the AI system learn from millions of hours of practice to improve on the foundation model.
The long story short of all these tech bro terms is robotic generality - the model will be able to control a robot to do every easy or medium difficulty task, the same way it can solve every easy or medium homework problem. This is what lets you automate (3), because you don’t need to do a lot of engineering work for a robot to do a million different jobs.
Multiple startups and deepmind are working on this.
- +2, You haven’t made the terms clear enough for there to even be a discussion.
- see above (placeholder for list formatting)
- Uh, OK? Then no (pure sneer: the plot thins). Robots building robots probably already happens in some sense, and we aren’t in the Singularity yet, my boy.
- Sure, why not.
(pure sneer response: imagine I’m a high school bully, and that I assault you in the manner befitting someone of my station, and then I say, “How’s that for a thought experiment?”)
Just to engage with the high school bully analogy, the nerd has been threatening to show up with his sexbot bodyguards that are basically T-800s from terminator for years now, and you’ve been taking his lunch money and sneering. But now he’s got real funding and he goes to work at a huge building and apparently there are prototypes of the exact thing he claims to build inside.
The prototypes suck…for now…
The thing about AI designing and building robots is that making physical things is vastly more expensive than pooping out six-fingered portrait jpegs. All that trial-and-error learning would not come cheap. Even if the AI were controlling CNC machining centers.
There’s no guarantee that the AI would have access to enough parts and materials to be able to be trained to a level of sufficient competence.
