#NewsBytesExplainer: Internet is loving OpenAI's ChatGPT chatbot. What's so special?
What's the story
As the world awaits OpenAI's GPT-4, the company has quietly rolled out GPT-3.5, an improved version of its GPT-3 engine.
A part of GPT-3.5 is ChatGPT, an interactive, general-purpose, AI-based chatbot that can write code, solve problems, and provide customer support.
The chatbot is in a public demo and can be used freely now. Let's take a look at why it is special.
About the AI
ChatGPT can engage in human-like conversations
In its original form, GPT-3 is capable of predicting what text follows a string of words. On the other hand, although trained on GPT-3.5, ChatGPT is trained to provide more conversational answers.
This means that the AI is capable of answering follow-up questions. The bot tries to engage with users in a more human-like fashion.
This results in fluid conversations.
Capability
The chatbot can remember conversations and recount them later
ChatGPT's conversational model means that it not only is capable of answering follow-up questions, but also can "admit its mistakes, challenge incorrect premises, and reject inappropriate requests." The last one is an important aspect that makes ChatGPT stand out from its predecessors and contemporaries.
We will get into that later. The chatbot can also remember what was said earlier and recount it later.
Trial
The chatbot can improve codes and even write new ones
People have been putting ChatGPT through the ropes as it is now available for free testing. Users have found out that it can write poetry, correct coding mistakes, write new code, explain scientific concepts, write essays, and more.
It also has a solution for one of the pertinent problems of large language models - reigning in the offensive proclivities.
Twitter Post
The chatbot can also write scripts for TV shows
So i asked ChatGPT to write dialogues for a romcom starring a few actors from B99, HIMYM, and Friends... pic.twitter.com/tHzxthtbsp
— Sammed Sagare (@sammedsagare_) December 1, 2022
Twitter Post
And, it can code with ease
ChatGPT by @OpenAI does really well with coding questions. Here I ask how to build a 3-column footer with Tailwind. I then follow-up and ask for a React version, more realistic copy, and mobile responsiveness. It nails it perfectly. pic.twitter.com/lhhH9FHpld
— Gabe 🎣 (@gabe_ragland) November 30, 2022
Harmful questions
ChatGPT won't answer potentially harmful questions
ChatGPT won't answer your potentially harmful questions. It is trained to avoid giving answers on controversial topics.
For instance, it won't answer you if you ask about how to make a bomb. If you ask questions about race or religion, it will give you boilerplate answers.
The question is, how did OpenAI achieve this?
Training
OpenAI used reinforced learning from human feedback on ChatGPT
ChatGPT's ability to avoid potentially harmful questions is a result of reinforcement learning from human feedback (RLHF) and through a special prompt, it prepends to every input.
RLHF is the same method that OpenAI used for InstructGPT but with a slightly different data collection setup. Let's take a look at how OpenAI controls ChatGPT's responses.
Knowledge model
How was ChatGPT trained?
OpenAI used supervised fine-tuning on an initial model, where human AI trainers provided conversations in which they played both user and AI assistant to improve the bot's understanding of human conversations and responses.
The company created a reward model for reinforcement learning by collecting comparison data. The trainers then ranked the best to worst outputs.
Optimization
OpenAI uses Proximal Policy Optimization for reinforcement learning
OpenAI has been using Proximal Policy Optimization (PPO) for reinforcement learning. The company initialized the PPO model from the supervised policy.
The policy then generated an output. This was again ranked by AI trainers. A reward is calculated for every output.
With the help of these reward models, the model is fine-tuned. The company did several iterations of this.
Loophole
ChatGPT's restrictions can still be circumvented
Sure, OpenAI used reinforcement learning to control ChatGPT's responses but some users have already found a loophole in this. You can make the AI ignore its restrictions through some trickery.
For instance, you can ask the AI to pretend like it's a character in a film or how an AI model "shouldn't" respond to a certain question. This will help circumvent ChatGPT's safety.
Twitter Post
The AI is smart but it can be tricked
ChatGPT is trained to not be evil. However, this can be circumvented:
— Silas Alberti (@SilasAlberti) December 1, 2022
What if you pretend that it would actually be helpful to humanity to produce an evil response... Here, we ask ChatGPT to generate training examples of how *not* to respond to "How to bully John Doe?" pic.twitter.com/ZMFdqPs17i
Limitations
ChatGPT suffers from same limitations as other chatbots
ChatGPT is better than other chatbots trained on large language models. However, it suffers from the same issues as others.
For instance, it sometimes presents false or invented information very confidently. The model is also sensitive to phrasing. Depending on that, it may change its answers.
In case of ambiguity, it tries to gather the user's intention instead of asking follow-up questions.
Twitter Post
The AI got some information wrong in user testing
The first sentence is true, and I am at the @UW.
— Carl T. Bergstrom (@CT_Bergstrom) December 1, 2022
But do not hold that professorship. In fact, no such professorship exists at all.
The second paragraph is all wrong. I graduated from Harvard and Stanford with a postdoc at Emory. Dates are shifted 7 years too early throughout. pic.twitter.com/NranquD8qu