What are the creators of Transfomer architecture doing now
Google may not have had the best start to the generative AI race. It is currently trailing OpenAI. But do you know OpenAI should thank Google for its success? Without Google's Transformer architecture, this AI race may not have happened. A 2017 paper by a few Google Brain scientists is the beginning of Transformer architecture. What are these scientists up to now?
Why does this story matter?
Transformer architecture is one of the most important things to happen to natural language processing (NLP) in the last decade. It is the foundation for most of the current leading AI models. Be it GPT-4, LaMDA, or PaLM, we can see the Transformer architecture behind them. It is highly stable and optimizable, leading to a paradigm shift in the AI space.
The paper about Transformer architecture was authored by 8 people
"Attention Is All You Need" is the name of the paper that changed AI research and development. The paper was authored by Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszekoreit, Lilian Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Most of these authors are not working at Google right now. In fact, seven of them left the company.
Vaswani and Parmar co-founded Adept AI Labs
Vaswani was one of the main brains behind the Transformer architecture. A graduate of the University of South Carolina, he was a staff research scientist at Google Brain from 2016 to 2021. In 2021, he left Google to start Adept AI Labs, an ML research and product lab. Niki Parmar, the only woman on the Transformer team, joined Vaswani in starting Adept.
They left Adpet and founded a start-up in stealth mode
Adept has raised over $400 million so far. The company is valued at over $1 billion. Vaswani was the chief scientist, while Parmar was the chief technology officer. Interestingly, neither of them is with the company now. Both left Adept in November 2022. In December, they co-founded a stealth start-up. We don't know much about it, though.
Shazeer co-founded Character.AI alongside another Google engineer
Like Vaswani and Parmar, Shazeer too left Google in 2021. He was at the company for over two decades. In 2021, he co-founded Character.AI with fellow former Google engineer Daniel De Freitas. The company has raised around $200 million so far. The company has around 20 employees. Character.AI is a chatbot web application powered by a neural language model.
Uszkoreit co-founded a company working on biological software
Uszekoreit is considered by many as the man behind discovering/inventing the Transformer architecture. He too left Google in 2021 after spending over 13 years at the company. In July 2021, he co-founded Inceptive, a company working on designing the next generation of RNA molecules with neural nets. The company has raised $20 million so far.
Jones is still at Google
Jones is the only one out of the eight authors still working at Google. He was the one behind the paper's title. Gomez left Google in 2019. In the same year, he founded Cohere, a company that provides access to LLMs and NLP tools through an API. It has raised about $400 million so far. Gomez is also the CEO of Cohere.
Polosukhin co-founded Ethereum-rival NEAR Protocol
Kaiser left Google in 2021 to join rival OpenAI. He was one of the co-creators of TensorFlow, Google's open-source ML platform. Polosukhin left Google the same year the paper was published (2017). He co-founded NEAR Protocol, a high-performance blockchain, in 2017. The company is currently valued at $2 billion and has raised $375 million.
