In News:

At the India-AI Impact Summit 2026, Bengaluru-based startup Sarvam AI unveiled two indigenous Large Language Models (LLMs)** trained on 35 billion and 105 billion parameters. These models are designed to be less power- and compute-intensive, while demonstrating improved performance in Indian languages. The development marks a significant milestone in India’s quest for sovereign and cost-efficient AI systems aligned with domestic needs.

Understanding Large Language Models (LLMs)

A Large Language Model (LLM) is an AI system built using transformer-based neural networks trained on massive text datasets to understand and generate human language.

They contain billions of parameters—internal variables learned during training—that help the model predict the next word in a sequence and generate coherent text.

How LLMs Work

1. Tokenisation: Text is broken into smaller units called tokens (word pieces or characters).
2. Embeddings & Transformer Architecture: Tokens are converted into numerical vectors. The self-attention mechanism helps the model determine which words in a sentence are contextually important, even if they are far apart.
3. Next-Token Prediction: The model generates language by predicting one token at a time based on probability distributions.
4. Layered Learning: Multiple transformer layers refine linguistic and semantic understanding—from grammar to reasoning patterns.

Stages of Training LLMs

1. Data Collection & Pre-processing

• Massive datasets sourced from books, websites, code repositories, etc.
• Cleaning to remove bias, spam, duplicates, and harmful content.
• Quality of data directly influences performance.

2. Pre-training (Self-Supervised Learning)

• Model learns via next-token prediction.
• Produces a base model capable of understanding grammar, facts, and reasoning.

3. Supervised Fine-Tuning

• Trained on curated prompt–response pairs.
• Enhances instruction-following ability and task performance (summarization, translation, Q&A).

4. Alignment via RLHF

• Reinforcement Learning from Human Feedback (RLHF).
• Humans rank outputs based on safety and quality.
• A reward model optimises responses to align with human values.

Challenges in Training LLMs in India

• Data Scarcity in Indian Languages
  • English and East Asian languages dominate internet data.
  • Indian languages remain underrepresented.
  • Many models rely on translation into English, increasing token consumption and cost.
• High Capital and Compute Costs
  • Requires clusters of Graphics Processing Units (GPUs).
  • Training costs run into millions of dollars.
  • Limited domestic venture capital for foundational AI research.
• Limited Immediate Commercial Use Cases: Training large models without clear monetisation pathways deters investment.
• Infrastructure Constraints
  • Dependence on imported high-end chips.
  • Energy-intensive training processes.

Innovation: Mixture of Experts (MoE) Architecture

Earlier LLMs activated all parameters during inference, making them computationally expensive.

The Mixture of Experts (MoE) architecture activates only a subset of parameters (“experts”) for each query.

Advantages:

• Reduced computational load
• Faster inference
• Lower electricity consumption
• Cost-efficient deployment in resource-constrained settings

Sarvam’s 105B parameter model leverages MoE to balance performance with efficiency, focusing on accuracy and Indian context alignment rather than sheer scale.

IndiaAI Mission: Government Support for Domestic AI

Launched in March 2024 with an outlay of ?10,372 crore, the IndiaAI Mission aims to build a comprehensive AI ecosystem.

Key Components:

• Compute Infrastructure
  • Over 36,000 GPUs commissioned in Indian data centres.
  • Additional 20,000 GPUs being added.
  • Target: 100,000 GPUs by end of 2026.
• Subsidised Access
  • Sarvam AI granted 4,096 GPUs from a common compute cluster.
  • Subsidy estimated at nearly ?100 crore.
  • Cluster cost approximately ?246 crore.
• Support for Innovation
  • Promotion of sovereign foundational models trained on Indian datasets.
  • Financial support covering compute and training costs.
  • Encouragement of open-source innovation.
• Talent Development
  • Training support for over 13,500 students.
  • Establishment of India Data and AI Labs.

Other Indian Efforts

• BharatGen (IIT Bombay-incubated): Multilingual 17B parameter model, targeted at sectors like education and healthcare.
• Gnani.ai: Small text-to-speech model.
• Indigenous focus on domain-specific and language-specific AI models.

In News:

At the AI Impact Summit, Bengaluru-based startup Sarvam AI unveiled two indigenous large language models (LLMs) under the collective name “Vikram.” The development coincides with global technology firms such as Nvidia and OpenAI announcing partnerships with Indian industry and academic institutions to strengthen India’s artificial intelligence (AI) ecosystem.

This marks a significant step in India’s efforts toward building Sovereign AI capabilities.

Sarvam AI

• Founded in 2023, headquartered in Bengaluru.
• Objective: Develop foundational AI models tailored to Indian languages and socio-economic contexts.
• Focus on multilingual and multimodal AI suited for India’s governance and development needs.

Key Features of the “Vikram” Models

• India-Specific Optimisation
  • Designed for Indian administrative, social and economic applications.
  • Strong performance in Indian language document processing and speech recognition.
• Multimodal Capabilities
  • Integrates text, speech and visual understanding.
  • Enables use cases in governance, healthcare, education and digital services.
• Edge and Offline AI Models
  • Models capable of running on devices without continuous internet connectivity.
  • Supports translation, speech-to-text and other AI functions locally.
  • Reduces dependence on cloud-based foreign infrastructure.

Sovereign AI: Meaning and Significance

Sovereign AI refers to the development of AI systems using domestically controlled infrastructure, data and expertise, covering the entire lifecycle - from data collection and model training to deployment and governance.

It aims to:

• Reduce dependence on US/China-based AI platforms.
• Ensure data security and privacy.
• Promote cultural and linguistic relevance.
• Strengthen national security and strategic autonomy.

Large Language Models (LLMs)

• LLMs are AI systems based on deep learning techniques.
• Trained on large volumes of unstructured data.
• Capable of understanding, summarising, generating and predicting text.
• Use probabilistic modelling to recognise patterns in language without explicit programming.

Potential Benefits for India

• Technological & Economic Gains
  • Boosts domestic innovation ecosystem.
  • Encourages startup growth and high-skill employment.
  • Enhances productivity across sectors.
• Human Capital Development
  • AI integration in education and skilling programmes.
  • Supports India’s demographic dividend with future-ready skills.
• Improved Governance
  • Multilingual AI can enhance e-governance delivery.
  • Applications in healthcare diagnostics, agriculture advisory, climate modelling and urban planning.
• Digital Inclusion
  • Bridges linguistic barriers for non-English speakers.
  • Expands participation in the digital economy.
• Strategic Autonomy: Reduces reliance on foreign AI infrastructure and proprietary systems.