Indian start-up's tech allows AI to run on regular PCs
What's the story
Indian AI start-up Ziroh Labs has developed a cost-effective system, called Kompact AI, in collaboration with the Indian Institute of Technology (IIT), Madras.
The innovative platform allows large-scale artificial intelligence (AI) models to run without high-end computing chips from companies like NVIDIA.
The unique part of Kompact AI is that it runs on standard CPUs of everyday computing devices instead of expensive GPUs that have been critical for AI growth.
Revolutionary approach
Kompact AI: A game-changer for AI development
Ziroh Labs' approach to AI development is largely centered around inference, the operation of AI systems after training.
The company claims it can optimize cutting-edge AI models for personal computer use.
In a recent demonstration, Ziroh Labs showcased their product running on a laptop with an Intel Xeon processor and querying models like Meta's Llama 2 and Alibaba's Qwen2.5.
Market impact
Ziroh Labs' technology has been tested by US chipmakers
Other tech companies have also used CPUs for some inference workloads. But, Ziroh Labs claims its approach delivers high-quality results.
The start-up's technology has even been tested by US chipmakers Intel Corp. and Advanced Micro Devices Inc., further establishing its efficacy.
"It's going to have a very profound market impact in the years ahead," said William Raduchel, former Chief Strategy Officer of Sun Microsystems and a tech advisor to the start-up.
Tech accessibility
Ziroh Labs' tech addresses AI access and affordability issues
Similar to other nations, in India too, developers have struggled to access and afford high-end NVIDIA chips required for building and supporting AI products.
"The AI divide is because only those with high-end expensive GPU-powered resources can access, develop and deploy powerful AI," said V. Kamakoti, Director at IIT Madras.
"We're demonstrating that you don't need a revolver to kill a mosquito," he added.