Possible End of Humanity from AI? Geoffrey Hinton at MIT Technology Review's EmTech Digital
16:39 
… in a world where people have bad intentions and where the political system is so broken that we can't even decide not to give assault rifles to teenage boys um if you can't solve that problem how are you going to solve this problem?
YOUTUBE sitHS6UDMJc Possible End of Humanity from AI? Geoffrey Hinton at MIT Technology Review's EmTech Digital
17:34
… I wish it was like climate change where you could say if you've got half a brain you'd stop burning carbon um it's clear what you should do about it it's clear that's painful but has to be done uh I don't know of any solution like that to stop these things taking over from us. What we really want I don't think we're going to stop developing them because they're so useful they'll be incredibly useful in medicine and in everything else um so I don't think there's much chance of stopping development. What we want is some way of making sure that even if they're smarter than us um they're going to do things that are beneficial for us that's called the Alignment Problem …
The goal of building safe, human-compatible AI, including potentially AGI/ASI is known as the Alignment Problem.
18:16 but we need to try and do that in a world where there's Bad actors who want to build robot soldiers that kill people and it seems very hard to me so I'm sorry I'm I'm sounding the alarm and
saying we have to worry about this and I wish I had a nice simple solution I could push but I don't but I think it's very important that people get together and think hard about it and see whether there is a solution it's not clear there is a solution so I mean talk to us about
No Technical Fix
19:11
They write programs and suppose you give them the ability to execute those programs which we'll certainly do …
For example, the ability of computational systems to establish Rules as genuine constraints where an analogous human legal system can only penalize violations makes possible Patterns of Organization that can only be approximated in society.