teaching_llm_applications

Sarvam

• Need for sovereign models

• 📝 Reading on AI sovereignty

• 📝 AI Sovereignty in the Global South: Power, Dependency, and Strategic Futures Simon Davies, Vikranth Harthikote Nagaraja, Innocent Nyalala, Nirav Bhatt, Soumya Banerjee Empowering Global South AI Workshop: The 40th Annual AAAI Conference on Artificial Intelligence 2026 and here

• 🎥📝 Listen to talk here

• How were Sarvam models built?

• Initially on top of Llama 2 architecture

• Key difference: vocabulary and training data

• 🎥 Video of Sarvam

• Trained on a diverse corpus of Indian languages, including Hindi, Tamil, Telugu, and Bengali.

• Improved tokenization for Indian languages
  • Standard tokenizers split Indian language words into smaller units, which reduces model efficiency.
  • Sarvam developed a custom tokenizer that can handle Indian language words more effectively.
• Fine-tuned for downstream tasks
  • Sarvam fine-tuned its models on various downstream tasks, including question answering, text summarization, and sentiment analysis.
  • These models are publicly available on Hugging Face.
• Open-source contribution
  • Sarvam released its models under the Apache 2.0 license, allowing researchers and developers to use them freely.

• now pretraining from scratch on Hopper series GPUs

• they also contribute to Llama open source repositories

This site is open source. Improve this page.