Possible End of Humanity from AI? Geoffrey Hinton at MIT Technology Review's EmTech Digital
YOUTUBE sitHS6UDMJc Possible End of Humanity from AI? Geoffrey Hinton at MIT Technology Review's EmTech Digital
3:00 
–
... tell us a little bit about what back propagation is and this is an algorithm that you you developed with a couple of colleagues back in the 1980s um many different groups discover back propagation ...
The special thing they did was to show that it could develop good internal representations and curiously they did that by implementing a tiny language model. It had embedding vectors that were only six components on the training set was 112 cases, but it was a language model. It was trying to predict the next term in our stray of symbols.
About 10 years later Joshua Benjo took basically the same net and used it on natural language it showed it actually worked for natural language if you made it much bigger
~
Neural Networks and Deep Learning page