This bodes well…
“We were training it in simulation to identify and target a Surface-to-air missile (SAM) threat. And then the operator would say yes, kill that threat. The system started realizing that while they did identify the threat at times the human operator would tell it not to kill that threat, but it got its points by killing that threat. So what did it do? It killed the operator. It killed the operator because that person was keeping it from accomplishing its objective,” Hamilton said, according to the blog post.
…huh
We trained the system–‘Hey don’t kill the operator–that’s bad. You’re gonna lose points if you do that’. So what does it start doing? It starts destroying the communication tower that the operator uses to communicate with the drone to stop it from killing the target.”
…this is perfectly fine, all weapon systems should have AI that are trained like gamers racking up kill count scores
Make sure they can self replicate and consume biomass for fuel. It’ll solve all the issues.
So US military trained AI acts like US military trained soldiers?
Trying to kill the people regulating them or exposing their actions?
:pinocchio-evil:
Military strategists playing Shadowrun rolls a critical fail and the DM made this headline
This definitely did not happen
It was via a simulated test, so no not physically. Sadly a drone did not blow up it’s operator in RL
I doubt that the AI model would be smart enough (in a simulation or otherwise) to understand the link between its “goals” and it’s instruction pathway. Like, there’s no way it would be programmed in a way to know that destroying the communications system would help get it points.
This is more than people would expect of a dog in terms of problem solving. Even simulated it did not happen.
2 options i can think of where it might be “real” Some dumb shit like role playing with a language model or something.
OR (if it’s real, this would be the most likely) a genetic algorithm that gradually learned that certain destroyed targets (that just happened to be the communications tower and the operator) increased its overall points so it evolved around targeting them. Which is what one would expect to happen.
