Meta Open-Sources Byte Latent Transformer LLM with Improved Scalability - InfoQ

Meta open-sourced Byte Latent Transformer (BLT), an LLM architecture that processes bytes instead of tokens. BLT dynamically groups bytes into patches, improving performance and robustness to noisy inputs, while allowing for simultaneous increases in model and patch size within a fixed inference budget. The BLT training and inference code are available on GitHub.