IIT Madras and Ziroh Labs to develop CPU-based AI platform for areas with accessibility challenges

Chennai: The Indian Institute of Technology Madras (IIT Madras), in association with the California-based startup Ziroh Labs, is establishing a new Centre of AI Research (COAIR) to address computing accessibility challenges in India. The COAIR will focus on developing practical, efficient AI solutions using CPU and edge device inferencing.

The initiative, launched on April 9 at IIT Madras, features ‘Kompact AI,’ a platform that enables AI models to run on standard CPUs, which are less expensive than the scarce Graphics Processing Units (GPUs).

Optimising 17 AI models to run on CPUs

Speaking during the demonstration, Prof V Kamakoti, director of IIT Madras, said, “Nature has taught us that one can effectively acquire knowledge and subsequently infer in only a limited set of domains. Attempts to acquire everything under the universe are not sustainable and bound to fail over a period.”

According to officials, Ziroh Labs has already optimised 17 AI models, including DeepSeek, Qwen and Llama, to run efficiently on CPUs. These models have undergone benchmarking with IIT Madras to evaluate their performance and accuracy.

No need of internet

The platform aims to democratise AI technology by making it accessible in resource-constrained environments.

Emphasising the need for ensuring the access to those with constraints, Dr Madhusudhanan B, principal consultant at IITM Pravartak Technologies Foundation, said, “By making AI models executable on CPUs, we are unlocking the potential of AI without the heavy infrastructure costs typically associated with GPUs.”

A key feature of Kompact AI is its ability to function without requiring internet connectivity, potentially allowing AI applications to reach rural and inaccessible areas across India. It uses multiple algorithmic optimisations and implementation strategies to execute models on low-end devices without requiring external communications like internet access.

Focus on smaller AI models

As part of this upcoming COAIR, IIT Madras will work on smaller AI models for specific use cases, using Kompact AI to optimise execution, improving the efficiency, accessibility and real-world application relevance of high-performance AI.

Prof Sadagopan, former director of IIIT-Bangalore, highlighted the importance of the project in aligning with the theme of ‘AI for All.’

“By combining affordability and availability of AI development and deployment tools using CPUs without excessive dependence on GPUs, Kompact AI proves that India can do frugal innovation at the cutting edge of technology. Even more interesting is the fact that such a solution using far less electrical power is sustainable,” he said.

The COAIR will focus on developing smaller, task-specific AI models for practical applications like aiding farmers, supporting students or assisting frontline workers in rural areas—use cases that don’t require the massive computing resources of general-purpose AI systems.