Artificial intelligence is becoming predominant in the technology industry in present times.
Whether it’s teaching new languages in a personalized way or a more efficient bureaucracy or creating efficient healthcare, AI chatbot is just getting better. Microsoft’s “emotional recognition” web app, Google’s surreal Deep Dream images are noteworthy examples of how big consumer-facing companies are playing up.
Intelligence is whatever machines haven't done yet. And even with tasks computers can beat, they aren’t doing it by replicating human intelligence.-Larry Tesler, American Computer Scientist
To understand the concept of Artificial Intelligence better we can conceive it to have three levels. Neural networks at the bottom level — they’re a type of computer architecture onto which AI is built. Machine learning is the middle level which can be explained as a program one might run on a neural network, training computers to look for certain answers in pots of data. Lastly, deep learning is the top level which is basically making the computer absorb gobs of information from videos across the Internet.
Over the past couple of decades, people have tried all sorts of different methods to try to teach computers. These methods include, for example, reinforcement learning, which is a type of machine learning technique that enables an agent to learn in an interactive environment by trial and error using feedback from its own actions and experiences. It is a part of the deep learning method that helps you to maximize some portion of the cumulative reward. It helps to attain a complex objective or maximize a specific dimension over many steps.
On the other hand, deep learning is another teaching method that analyzes data at different abstractions. So, if a deep learning system is looking at a picture, each layer is essentially tackling a different magnification. The bottom layer might look at just a 5 x 5 grids of pixels, answering simply “yes” or “no” as to whether something shows up in that grid. If it answers yes, then the layer above looks to see how this grid fits into a larger pattern. Is this the beginning of a line, for example, or a corner? This process gradually builds up, allowing the software to understand even the most complicated data by breaking it down into constituent parts. This is a long, iterative process, with the system slowly getting better based on feedback
As computer scientists, Larry Tesler put it: “Intelligence is whatever machines haven’t done yet.” And even with tasks computers can beat, they aren’t doing it by replicating human intelligence. “When we say the neural network is like the brain it’s not true,” says LeCun. “It’s not true in the same way that airplanes aren’t like birds. They don’t flap their wings, they don’t have feathers or muscles.” If we do create intelligence, he says, it “won’t be like human intelligence or animal intelligence. It’s very difficult for us to imagine, for example, an intelligent entity that does not have [the impulse towards] self-preservation”.
For decades, machines have struggled with the subtleties of human language, and even the recent boom in deep learning powered by big data and improved processors has failed to crack this cognitive challenge. Algorithmic moderators still overlook abusive comments, and the world’s most talkative chatbots can barely keep a conversation alive.
To overcome this, supervised learning is at the core of most of the recent success of machine learning. However, it can require large, carefully cleaned, and expensive to create datasets to work well. Unsupervised learning is attractive because of its potential to address these drawbacks. Since unsupervised learning removes the bottleneck of explicit human labeling it also scales well with current trends of increasing compute and availability of raw data.
OpenAI’s new algorithm, named GPT-2 (GPT stands for “generative pre-trained transformer.”) launched a year back, is one of the most exciting examples yet. It excels at a task known as language modeling, which tests a program’s ability to predict the next word in a given sentence. Give it a fake headline, and it’ll write the rest of the article, complete with fake quotations and statistics. The latest invention in this series of autocomplete tools designed by OpenAI is GPT-3. Like all deep learning systems, GPT-3 looks for patterns in data. To simplify things, the program has been trained on a huge corpus of text that it’s mined for statistical regularities. These regularities are unknown to humans, but they’re stored as billions of weighted connections between the different nodes in GPT-3’s neural network. It’s hard to estimate the total size, but we know that the entirety of the English Wikipedia, spanning some 6 million articles, makes up only 0.6 percent of its training data The rest comes from digitized books and various web links.
That means GPT-3’s training data includes not only things like news articles, recipes, and poetry, but also coding manuals, fan fiction, religious prophecy, guides to the songbirds of Bolivia, and whatever else you can imagine. Any type of text that’s been uploaded to the internet has likely become grist to GPT-3’s mighty pattern-matching mill. And, yes, that includes the bad stuff as well. It’s all there, feeding the machine.
Different features of GPT-3 are that it can release short stories, songs, technical manuals or can imitate writing styles of different authors. However a lot of fine-tuning is required to remove the hateful and racist language.
In other words, GPT-3 is like a huge, varied scrapbook that combines text from all over. Although there’s a bigger journey to cover in the language generating AI system, GPT-3 can be considered as a giant leap.