Microsoft's GPT-3 technology could soon write code using ordinary language
What's the story
A year after funneling $1 billion in OpenAI and gaining exclusive licensing rights to GPT-3, on May 25, Microsoft shared its plans to bring the GPT-3 natural language processing (NLP) technology to programming applications.
This first commercial application of GPT-3, since Microsoft acquired its rights, was revealed by Microsoft boss Satya Nadella at the Build 2021 conference for developers.
Here are more details.
What’s GPT?
GPT-3's NLP system is trained using over 500 billion words
GPT-3 can be described as an NLP system that works with Generative Adversarial Network-based (GAN) machine learning to make evolutionary changes at a rapid rate. This exponentially improves the words it suggests each time.
Additionally, it works using a data set of over 500 billion words used on the internet. With machine learning, larger data sets help improve prediction accuracy at a faster rate.
Code that writes itself
GPT-3's application to programming could empower amateur programmers: Microsoft VP
Describing GPT-3's benefit for programmers, Satya Nadella said, "If you can describe what you want to do in natural language, GPT-3 will generate a list of the most relevant formulas for you to choose from."
Microsoft VP Charles Lamanna told WIRED that GPT-3 can empower people with little coding experience. It will translate natural language into PowerFX, a programming language akin to Excel commands.
Best in class?
GPT-3's neural network architecture is built using 175 billion parameters
Microsoft's GPT-3 is based on a neural network architecture called Transformer. It is also used by Baidu, Google, Microsoft, Salesforce, and NVIDIA to create data sets for training the NLP model using data from the web.
The largest version of Google's 2018 language model called BERT has 340 million parameters to build neural networks with. Meanwhile, one-year-old GPT-3 uses a whopping 175 billion parameters.
Research
GPT-3's AI capabilities aren't yet fit to replace human programmers
However, that doesn't mean GPT-3 is all set to replace programmers entirely. WIRED reported that a group of AI researchers led by Dan Hendrycks found that the best learning models succeeded only 14% of the time on introductory programming challenges.
Hendrycks reportedly thinks that such models (including Microsoft's GPT-3) will address specific tasks before tackling more generalized forms of coding.
Present functionality
GPT-3 helps programmers use natural language to create PowerFx commands
Microsoft's blog post explained that a user building an e-commerce platform could simply describe "find products where the name starts with 'kids'". The GPT-3 model would then convert the requirement into a PowerFx formula such as "Filter('BC Orders' Left('Product Name',4)="Kids")".
Since PowerFx is an open-source programming language of the Power platform, Microsoft plans to infuse it into other tools, too.
Coming soon
AI-powered features will be previewed for Power Apps by June-end
Microsoft noted that when PowerFx gets integrated into other tools using the Power Platform, NLP features using GPT-3 would also follow suit.
The company said that AI-powered features in Power Apps will be available for preview by the end of June.
This significant development explains why Microsoft CEO Satya Nadella was so excited to share major changes for developers when Build 2021 began.