Compute Optimal Tokenization

Published in TMLR, 2026

Recommended citation: Tomasz Limisiewicz, Artidoro Pagnoni, Srini Iyer, Mike Lewis, Sachin Mehta, Alisa Liu, Margaret Li, Gargi Ghosh, and Luke Zettlemoyer (2026). "Compute Optimal Tokenization." TMLR (accepted). Paper