Thamme Gowda

From O(N) to O(log N): A Faster BPE Training Algorithm, Buried and Rediscovered

Mon, 30 Mar 2026 20:30:00 +0000

I wrote a fast BPE training algorithm in 2020, buried it in a Python codebase, and forgot about it. Five years later, I rewrote it in C++ and benchmarked it: up to 11× faster than SentencePiece. The trick? A max-heap with lazy deletion instead of periodic linear scans.

Building a Jinja2 Template Engine from Scratch in C++

Tue, 10 Mar 2026 12:00:00 +0000

A tutorial on building a Jinja2 template engine in C++ for rendering LLM chat templates. Covers the lexer, recursive descent parser, and tree-walking evaluator, with real examples from HuggingFace model templates.

I Let Two AI Agents Race to Modernize pigz

Sat, 07 Mar 2026 20:20:00 +0000

I gave Claude Opus 4.6 and GPT 5.4 the same task: rewrite pigz in modern C++23 as a thread-safe library. One agent did a clean-room rewrite, the other wrapped the legacy code. The winner went on to beat pigz by up to 1.8x compression and 2.4x decompression.

Sequence Transduction: Generalization and Challenges

Tue, 04 May 2021 10:20:00 +0000

Sequence to sequence transduction is a general problem, for which many other problems are special cases. I also highlight some challenges of this general problem.

Many-to-English Machine Translation Tools, Data, and Pretrained Models

Sun, 25 Apr 2021 10:20:00 +0000

We present useful tools for machine translation research: MTData, NLCodec, and RTG. We demonstrate their usefulness by creating a multilingual neural machine translation model capable of translating from 500 source languages to English. We make this multilingual model readily downloadable and usable as a service, or as a parent model for transfer-learning to even lower-resource languages.

Macro-Average: Rare Types Are Important Too

Thu, 11 Mar 2021 10:20:00 +0000

We explore the simple type-based classifier metric, maf1, and study its applicability to MT evaluation. We find that MacroF1 is competitive on direct assessment, and outperforms others in indicating downstream cross-lingual information retrieval task performance.

Finding the Optimal Vocabulary for Neural Machine Translation

Sun, 01 Nov 2020 10:20:00 +0000

We cast neural machine translation (NMT) as a classification task in an autoregressive setting and analyze the limitations of both classification and autoregression components. Classifiers are known to perform better with balanced class distributions during training. Since the Zipfian nature of languages causes imbalanced classes, we explore its effect on NMT.

Notes

Mon, 01 Jan 0001 00:00:00 +0000

Here are some useful notes I have collected:

My Notes

Python Best Practices: Get PDF; Google Slides
Introduction to Quantum Optimization using D-WAVE 2X
2019-Fall CSCI-662
- Google Cloud Setup for Pytorch with GPU
- Non-Linear Classifiers
SLURM 101
Unsupervised NMT Summary

Notes From Literature

Tools for Sceptical Thinking from the book The Demon-Hunted World by Carl Sagan
Pale Blue Dot by Carl Sagan
Creative Thinking by Claude Shannon, 1952

Publications

Mon, 01 Jan 0001 00:00:00 +0000

Software

Mon, 01 Jan 0001 00:00:00 +0000

Solving problems using math and computers is my favourite job to do. In early days of my career, I aspired to be a good software engineer, and I passionately pursued it until I slowly transitioned towards a research career. While I write less code as a researcher than what I’m used to doing as a software engineer, I emphasize on good software engineering practices, and open sourcing of tools with a permissible license. In the beginning (2012-2016) I wrote much of my code in Java/Groovy/Scala, but in the recent years (2016-Now), Python has become my go to choice. I have released a bunch of tools to PyPi.