Shashi Kumar

Ph.D. Student — EPFL & Idiap Research Institute, Switzerland

I am a fourth-year Ph.D. student at École Polytechnique Fédérale de Lausanne (EPFL), Lausanne, Switzerland and Idiap Research Institute, Martigny, Switzerland, advised by Prof. Andrea Cavallaro and Prof. Petr Motlíček. My research aims to improve the reasoning, efficiency, and robustness of LLMs and multimodal LLMs. Before the Ph.D., I worked in industry training and deploying production ASR systems at scale at Samsung Research and Level AI.

Email Scholar LinkedIn CV
Shashi Kumar

Research

Latent Reasoning

Reasoning in continuous space instead of through explicit tokens. GLR (NeurIPS 2026) frames latent reasoning as a geometric path in the LLM's token-embedding space, anchored by chain-of-thought traces, and yields much shorter generations.

RL-based Post-training

Post-training with reinforcement learning. Using GRPO, we adapt SpeechLLMs on synthetic speech alone and reduce WER by 40% relative to supervised fine-tuning (SLT 2026).

Test-Time Compute Scaling

Trading inference-time compute for accuracy. LARM, a depth-conditioned looped Transformer, reuses one shared block across recurrent passes so that encoder depth can be scaled at inference time.

Parameter-Efficient Fine-Tuning

FVAE-LoRA (NeurIPS 2025) factorizes task-relevant and residual features in low-rank adapters, improving robustness and performance across image, text, and audio tasks.

Cross-Modal Alignment with Optimal Transport

OTTC is a differentiable sequence-alignment framework based on 1D optimal transport, enabling the model to learn a single alignment and perform ASR in an end-to-end manner.

Multitask & Streaming ASR

Single models that jointly handle transcription, speaker change detection, endpointing, and NER (TokenVerse, EMNLP 2024; TokenVerse++, ASRU 2025), streaming ASR with self-supervised encoders (XLSR-Transducer), and compressed conversational context for LLM-based ASR (SLT 2026).

Currently working on interpretability of cross-modal interactions.

News

Selected Publications

For a full list, see my Google Scholar profile. * equal contribution.

2026 NeurIPS
Geometric Latent Reasoning Induces Shorter Generations in LLMs
S. Kumar*, Y. Kaloga*, P. Motlicek, I. Kodrasi, and A. Cavallaro
Framed latent reasoning as a geometric path in the LLM's token-embedding space, with chain-of-thought traces as anchors. On math benchmarks with Qwen3, GLR needs about 80% fewer generation steps and runs 5.3× faster than explicit chain-of-thought.
[Paper] [Website]
2026 arXiv Preprint
Test-Time Compute Scaling for ASR with Depth-Conditioned Looped Transformers
Y. Kaloga*, S. Kumar*, S. A. Sheikh*, D. Khalil, P. Motlicek, and I. Kodrasi
Introduced LARM, a looped Transformer encoder that reuses one shared block across recurrent passes, so encoder depth can be scaled at inference time. On LibriSpeech, it outperforms a 16-block encoder with nearly 4× fewer parameters.
[Paper] [Code]
2026 IEEE SLT
When Synthetic Speech Is All You Have: Better Call GRPO
S. Kumar*, Y. Labrak*, H. Watawana, S. Burdisso, E. Villatoro-Tello, K. Hacioğlu, P. Motlicek, and A. Stolcke
Applied reinforcement learning with GRPO to SpeechLLMs adapted on synthetic speech only. WER drops by 40% relative to supervised fine-tuning, and by 45% when GRPO follows SFT.
[Paper] [Website]
2026 IEEE SLT
Distilling Conversations: Abstract Compression of Conversational Audio Context for LLM-based ASR
S. Kumar, E. Villatoro-Tello, S. Burdisso, K. Hacioglu, T. Bañeras-Roux, H. Watawana, D. Sanchez-Cortes, S. Madikeri, P. Motlicek, and A. Stolcke
Showed that conversational context mainly helps LLM-based ASR recognize named entities, and introduced Abstract Compression, which replaces prior-turn audio with a fixed number of learned latent tokens, recovering part of the benefit of raw context with a much smaller audio footprint.
[Paper]
2025 NeurIPS ★ Idiap PhD Paper Award
Latent Space Factorization in LoRA
S. Kumar, Y. Kaloga, J. Mitros, P. Motlicek, and I. Kodrasi
Proposed FVAE-LoRA, a parameter-efficient fine-tuning method that separates task-relevant and residual features, improving robustness across image, text, and audio tasks.
[Paper] [Code]
2025 arXiv Preprint
A Differentiable Alignment Framework for Sequence-to-Sequence Modeling via Optimal Transport
Y. Kaloga*, S. Kumar*, P. Motlicek, and I. Kodrasi
Introduced a novel differentiable sequence-alignment framework based on 1D optimal transport, enabling end-to-end ASR with learned alignments.
[Paper] [Code]
2025 ICASSP — SALMA Workshop ★ Best Paper Award
Performance Evaluation of SLAM-ASR: The Good, the Bad, the Ugly, and the Way Forward
S. Kumar, I. Thorbecke, S. Burdisso, E. Villatoro-Tello, M. K E, K. Hacioglu, P. Rangappa, P. Motlicek, A. Ganapathiraju, and A. Stolcke
Evaluated SLAM-ASR across domain shifts and speech perturbations, identifying robustness gaps and providing guidance for reliable LLM-based ASR.
[Paper]
2025 ICASSP
XLSR-Transducer: Streaming ASR for Self-Supervised Pretrained Models
S. Kumar, S. Madikeri, J. Pablo Zuluaga Gomez, E. Villatoro-Tello, I. Thorbecke, P. Motlicek, M. K E, and A. Ganapathiraju
Enabled streaming ASR with self-supervised encoders, outperforming a streaming Zipformer by 42% relative WER. Attention sinks halve the left context while improving WER by 12% relative.
[Paper]
2024 EMNLP
TokenVerse: Towards Unifying Speech and NLP Tasks via Transducer-based ASR
S. Kumar, S. Madikeri, J. Pablo Zuluaga Gomez, I. Thorbecke, E. Villatoro-tello, S. Burdisso, P. Motlicek, K. Pandia D S, and A. Ganapathiraju
A transducer-based framework unifying transcription, speaker change detection, endpointing, and NER in a single model, outperforming cascaded pipelines.
[Paper]

Experience

May 2022 – Apr 2023

Senior ML Engineer → Lead ML Engineer

Level AI, Remote, California, USA
  • Led low-resource conversational ASR using domain adaptation and entity-aware transducers.
  • Built the production VAD and speaker diarization pipeline, outperforming the vendor system.
  • Scaled ASR deployment to 120K+ audio hours/month across multiple customer tenants.
Jun 2017 – Apr 2022

Software Engineer → Lead Software Engineer

Samsung Research, Bangalore, India
  • Developed hybrid and end-to-end ASR for accented and far-field speech for Bixby.
  • Built VAE-based speaker normalization for far-field ASR; deployed and published at Interspeech.

Awards & Service

Best Paper Award — SALMA Workshop at ICASSP 2025
Idiap PhD Paper Award — for "Latent Space Factorization in LoRA" (NeurIPS 2025)
Challenges: DiSPLACE-M Challenge 2026: team ranked 1st in 3 of 4 tracks, including diarization and Hindi Devanagari ASR for medical conversations. [Paper]
Challenges: Interspeech DiCOVA Challenge 2021: ranked 5th out of 29 teams in a COVID-19 detection task using cough sounds (AUC 83.93). [Paper]
Reviewer: NeurIPS, ICLR, ICASSP, Interspeech, EUSIPCO, and ACL Rolling Review (ARR).
Presentations: NeurIPS 2025, Interspeech 2025, and EMNLP 2024.

Education

Aug 2023 – Present

Doctor of Philosophy (Ph.D.)

EPFL, Lausanne & Idiap Research Institute, Martigny, Switzerland
Advisors: Prof. Andrea Cavallaro, Prof. Petr Motlíček · Expected: Aug 2027
Industry collaboration with Uniphore, USA, on SpeechLLMs.
Aug 2013 – May 2017

Bachelor of Technology (B.Tech.)

Indian Institute of Technology (IIT) Guwahati, India
Electronics and Communication Engineering (ECE)