Meta's AI, trained on 48mn scientific papers, dumped in 2-days
Meta AI in collaboration with Papers with Code, launched Galactica, an open-source large language model for science, on November 15. Their prime objective was to tackle the overload of scientific information. Galactica claimed that it could "store, combine and reason about scientific knowledge." Sadly, a lot went wrong with the online tool and it was taken down in just two days.
Why does this story matter?
Galactica could be a great advantage to researchers as it claimed to organize scientific literature. Its algorithm was said to be "high-quality and highly curated" but in reality, it was not so. Instead of "benefitting the scientific community," the search engine provided faulty results. The inaccuracy of Meta's AI model raises concerns over the effectiveness of AI-powered platforms, especially in the scientific domain.
Galactica was trained on over 48 million scientific papers
As per the official website, Galactica was a "powerful large language model (LLM) trained on over 48 million papers, textbooks, reference material, compounds, proteins, and other sources of scientific knowledge." "You can use it to explore the literature, ask scientific questions, write scientific code, and much more." The site also said that Galactica outperformed GPT-3, one of the most popular LLMs, by 68.2%.
Here's a demonstration of how Galactica worked
What went wrong with the tool?
The platform failed to perform kindergarten-level mathematics and gave the wrong answers. When Galactica was asked to summarise work by Julian Togelius, an associate professor at NYU, the tool got the name wrong and also failed to provide a summary of his work. According to Gary Marcus, a psychology professor at NYU, 85% of the results presented by Galactica about him were incorrect.
A University of Washington professor called it a "bullsh#t generator"
The Wikipedia article on 'Hanlon's razor' says something else
Bergstrom's Twitter post shows how Galactica failed to generate a Wiki article. "Hanlon's razor is an adage or rule of thumb that states never attribute to malice that which is adequately explained by stupidity," reads the official Wikipedia article on 'Hanlon's razor.' "It is a philosophical razor that suggests a way of eliminating unlikely explanations for human behavior."
Galactica displayed a warning message along with search results
The official website has a warning message in bold stating "Never follow advice from a language model without verification." Galactica even displayed the disclaimer "Outputs may be unreliable. Language Models are prone to hallucinate text," along with every result.
