Roblox Creates Its Own Language Model

Roblox, the popular online gaming platform, has introduced a groundbreaking feature that is set to revolutionize the way players communicate within the gaming community. On February 5th, the company's Technical Director, Dan Sturman, announced the creation of a custom Language Model (LLM) that enables real-time text translation among players speaking different languages. This innovation reduces message delay to just 100 milliseconds, making conversations seamless and instant.

While Roblox has already automated content translation, the company aimed to go beyond translating static content. Sturman highlighted two significant challenges in creating this translator: building a system capable of translating between all 16 supported languages independently and ensuring it was fast enough to support real-time chats. To tackle this, Roblox developed a unified LLM based on a transformer, handling all language pairs in a single model.

Roblox started by training its transformer-based LLM using both public and private data. Then, the company enlisted the help of "expert" translation apps to teach the model each individual language. Sturman explained that translating less common pairs, such as French to Thai, proved challenging due to limited high-quality data, requiring Roblox to use a "back translation" method for accuracy.

The model also underwent training to understand human slang. Roblox brought in evaluators to translate "popular and trendy terms" for each language. This was an ongoing process, with evaluators continuously updating the system with the latest phrases.

During testing, Roblox discovered that the new translation system significantly improved user engagement and session quality on its platform. With 70 million active users from over 180 countries worldwide, Roblox facilitates over 2.4 billion daily messages.

Roblox CEO David Baszucki expressed his vision for functional compatibility, believing that all metaverse users should have the ability to move non-fungible tokens (NFTs) and other digital assets across multiple independent platforms. Roblox's innovative language model is a major step towards realizing this ambitious goal.