November 15, 2026, (Inside AI) — India's push to build its own artificial intelligence models is running into a problem that computing power cannot solve: a severe shortage of digital data in the country's regional languages. On November 3, the IndiaAI Mission, under the Union Ministry of Electronics and IT, released three indigenous foundation language models. EKA-1 covers the 22 scheduled Indian languages and handles tasks like mathematics, coding, and reasoning. The other two, Dhīr 2B and Dhīr 5B, support 10 Indian languages for document retrieval and government and business services.
These models received subsidised access to national AI compute infrastructure and were built under the IndiaAI Innovation Centre pillar, set up in 2024 to foster homegrown large language models (LLMs) and domain-specific foundational models. But compute is only half the battle. The other half is data, and for many of India's languages, there is precious little of it.
The stakes are high. India is one of the most linguistically diverse countries on Earth, with speakers of over 19,500 languages and dialects. Without multilingual AI models, large populations risk relying on poor tools and losing access to digital services in health, education, agriculture, and governance. The new models are a start, but they expose a deeper structural gap: the divide between high-resource and low-resource languages.
Why Bhojpuri Speakers Still Lack AI Tools
A language's resource status depends not on how many people speak it but on how much digitised data exists. English, Hindi, and Arabic are high-resource: they have vast amounts of text, audio, and research available for training. Languages like Assamese, Bhojpuri, or Dogri are low-resource, with scant digital text and audio. Bhojpuri, despite being spoken by millions, remains low-resource because there is little capital for collecting linguistic data and building models.
Read: AI Chatbots May Narrow Human Knowledge, Study Finds
Building a multilingual LLM requires different data at different stages. Unlabelled text and speech from Wikipedia, news, and digital books help a model learn grammar, vocabulary, and sentence structure during pre-training. But raw data is only the first step. Thousands of hours of recordings and text must be annotated with structural markers, tags, and metadata so the model can understand context. This human labelling is labour-intensive and cannot be automated. It demands native speakers and domain experts who can capture local idioms, cultural nuances, and context.
Web crawling, often touted as a cheap fix, mostly benefits English and other high-resource languages that have historically dominated the internet. For Hindi, online content is comparatively sparse; for Assamese or Santali, it is far worse. What little exists in regional languages often comes from legislation, government documents, and news, which differ sharply from everyday spoken language. Relying solely on such sources strips away cultural context, producing models that can speak a language grammatically but lack cultural grounding.
Consider a Tamil chatbot asked about nutritious foods. A model trained on translated or web-scraped data might respond in perfect Tamil but suggest asparagus, because its dataset statistically associates "nutritious food" with that term. The translation is accurate, but the output is culturally irrelevant. In healthcare, education, farming, or local law, such misinformation can be serious, returning foreign legal advice or regionally irrelevant medical guidance.
Tokenisation adds another layer of cost. Popular models like ChatGPT or GPT-4 use tokenisers built for English, which struggle with Indic scripts. A sentence that takes 10 tokens in English might take 35 to 140 tokens in an Indian language. Commercial AI services charge per token, making regional language processing expensive and slow. Fixing this "token tax" requires native Indic tokenisers, which in turn demand rich, culturally grounded linguistic datasets.
The IndiaAI Mission's new models are a significant step, but they arrive amid a global race for AI sovereignty. Countries like France, Japan, and the UAE are also investing in native language models to reduce reliance on American and Chinese AI. India's effort is notable for its scale and linguistic diversity, but the data challenge is formidable. Unlike compute, which can be bought or subsidised, linguistic data must be painstakingly collected, annotated, and curated by communities themselves.
"Adding digital datasets to a low-resource language does not guarantee the language becoming high-resource overnight," notes a researcher at Carnegie India's Technology and Society Program. "Languages like English, Arabic, and Hindi are high-resource also owing to centuries of literature, writing and media." The researcher, who spoke on condition of anonymity, added that AI-driven data collection captures only a part of how languages are used.
Read: Cloudflare Launches Disallow AI Training Setting to Separate Search from AI Training
The path forward, according to policy analysts, involves collaboration with regional communities. When builders work with community members, those members can help digitise and preserve their languages, ensuring that indigenous knowledge systems do not fade in an internet-first economy. Such partnerships can also produce the culturally relevant data that models need to be truly useful.
For now, the IndiaAI Mission's models represent a proof of concept. But without sustained investment in data collection and community engagement, they may remain limited in their ability to serve India's diverse population. The next phase of India's AI journey will be less about compute and more about the painstaking work of building datasets that reflect the country's linguistic richness.
Sofia spent years studying how machines parse human language before pivoting to cover the systems now rewriting it. She reads technical papers the way others read news and looking for what the numbers are actually saying beneath the headlines.
