Microsoft's GPT-3 technology could soon write code using ordinary language
A year after funneling $1 billion in OpenAI and gaining exclusive licensing rights to GPT-3, on May 25, Microsoft shared its plans to bring the GPT-3 natural language processing (NLP) technology to programming applications. This first commercial application of GPT-3, since Microsoft acquired its rights, was revealed by Microsoft boss Satya Nadella at the Build 2021 conference for developers. Here are more details.
GPT-3's NLP system is trained using over 500 billion words
GPT-3 can be described as an NLP system that works with Generative Adversarial Network-based (GAN) machine learning to make evolutionary changes at a rapid rate. This exponentially improves the words it suggests each time. Additionally, it works using a data set of over 500 billion words used on the internet. With machine learning, larger data sets help improve prediction accuracy at a faster rate.
GPT-3's application to programming could empower amateur programmers: Microsoft VP
Describing GPT-3's benefit for programmers, Satya Nadella said, "If you can describe what you want to do in natural language, GPT-3 will generate a list of the most relevant formulas for you to choose from." Microsoft VP Charles Lamanna told WIRED that GPT-3 can empower people with little coding experience. It will translate natural language into PowerFX, a programming language akin to Excel commands.
GPT-3's neural network architecture is built using 175 billion parameters
Microsoft's GPT-3 is based on a neural network architecture called Transformer. It is also used by Baidu, Google, Microsoft, Salesforce, and NVIDIA to create data sets for training the NLP model using data from the web. The largest version of Google's 2018 language model called BERT has 340 million parameters to build neural networks with. Meanwhile, one-year-old GPT-3 uses a whopping 175 billion parameters.
GPT-3's AI capabilities aren't yet fit to replace human programmers
However, that doesn't mean GPT-3 is all set to replace programmers entirely. WIRED reported that a group of AI researchers led by Dan Hendrycks found that the best learning models succeeded only 14% of the time on introductory programming challenges. Hendrycks reportedly thinks that such models (including Microsoft's GPT-3) will address specific tasks before tackling more generalized forms of coding.
GPT-3 helps programmers use natural language to create PowerFx commands
Microsoft's blog post explained that a user building an e-commerce platform could simply describe "find products where the name starts with 'kids'". The GPT-3 model would then convert the requirement into a PowerFx formula such as "Filter('BC Orders' Left('Product Name',4)="Kids")". Since PowerFx is an open-source programming language of the Power platform, Microsoft plans to infuse it into other tools, too.
AI-powered features will be previewed for Power Apps by June-end
Microsoft noted that when PowerFx gets integrated into other tools using the Power Platform, NLP features using GPT-3 would also follow suit. The company said that AI-powered features in Power Apps will be available for preview by the end of June. This significant development explains why Microsoft CEO Satya Nadella was so excited to share major changes for developers when Build 2021 began.
