Earlier this year, Roblox unveiled its ambitious plan for adding generative Roblox AI into its platform, aiming to make content creation more streamlined for all users. The recent developments in generative AI, particularly in large language models (LLMs), promise a new horizon for enhancing immersive experiences. Such advancements not only speed up the creation process but also ensure a safer environment, all while keeping computational demands in check.
One notable trend in AI is the rise of multimodal models. These models, adept at processing a variety of content forms, from images and text to 3D models and audio, are redefining creation tools. Their ability to produce outputs that combine multiple modes — for example, text with corresponding visuals — stands out as a significant innovation. For Roblox, this represents a twofold advantage:
- It boosts the productivity of seasoned content creators.
- It democratizes the creative process, enabling a broader range of users to materialize their ideas on the platform.
At the recent Roblox Developers Conference (RDC), the company announced the launch of several tools incorporating generative AI into Roblox Studio. These tools aim to facilitate quicker scaling of projects, accelerate the iteration process, and enhance the creative capacities of users, ensuring top-notch content generation.
Roblox AI Assistant
Roblox has consistently furnished its creators with the necessary resources to craft detailed 3D experiences. Yet, many creators turned to third-party generative and conversational AI tools for assistance. Although beneficial in easing some burdens, these generic versions weren’t tailored for Roblox’s unique workflows or its distinct code, jargon, and vernacular. Consequently, creators encountered substantial challenges when trying to integrate these tools with Roblox content. Recognizing this gap, Roblox announced its effort to incorporate these tool’s benefits directly into Roblox Studio during the RDC, introducing an early version of their tool called “Assistant.”
“Assistant” is Roblox’s in-house conversational AI designed to streamline the creation process for creators, regardless of their experience level. It aims to minimize time spent on routine tasks, allowing creators to focus on areas like storyline, gameplay, and overall design. With its vast library of public 3D models and the capability to merge the model with platform APIs, Roblox stands out as a prime candidate to develop such an AI model for intricate 3D universes. With Assistant, creators can employ plain language prompts to design scenes, modify 3D models, and infuse objects with interactive traits. The tool covers three key creation stages: comprehension, programming, and construction:
Comprehension: Whether a novice or a Roblox expert, Roblox AI Assistant facilitates answers to diverse queries using plain language. Programming: Building on the existing “Code Assist” feature, Assistant can enhance coding, elucidate certain code segments, and offer debugging solutions. Construction: Creators can swiftly turn concepts into tangible models. By entering commands like “Illuminate this path with streetlights” or “Construct a varied forest and adorn with shrubs and blossoms,” entire scenes can materialize. Engaging with Assistant is envisioned as a dynamic, two-way process, allowing creators to give feedback and refine outcomes. It’s akin to collaborating with a seasoned co-creator, facilitating brainstorming and idea optimization.
During the RDC, Roblox made an additional announcement regarding their new Roblox AI tool, Assistant. Developers were presented with the opportunity to contribute their anonymized Luau script data to enhance the efficiency and capability of Roblox AI tools, such as Code Assist and Assistant. By doing so, they would be aiding not only the Roblox developer community but, if they chose to extend their contributions beyond Roblox, also helping the broader Luau developer community. This is because the shared script data would be incorporated into a dataset accessible to third parties, aiming to refine their AI chat tools for improved Luau code suggestions.
Streamlining the Avatar design
Roblox’s vision is for each of its impressive 65.5 million daily users to own an avatar that genuinely reflects and conveys their identity. While recently, the capability was introduced for UGC Program members to design and market avatar bodies and standalone heads, the current design process involves navigating Studio or the UGC Program. Coupled with a requisite advanced skill level and several days to integrate features like facial expressions and 3D rigging, avatar creation becomes a prolonged endeavor. As a result, the avatar options remain limited. However, Roblox envisions expanding beyond these limitations.
With the intent of facilitating all Roblox users in obtaining a distinct, dynamic avatar, the company aims to simplify the avatar creation process. At the RDC, the unveiling of a revolutionary tool scheduled for 2024 was announced. This innovation will allow the transformation of an image or multiple images into a custom avatar. Any creator affiliated with Studio or the UGC program can employ this tool, uploading an image and subsequently personalizing the generated avatar. Roblox’s long-term aspiration includes embedding these Roblox AI features directly within Roblox experiences.
To actualize this, Roblox is channeling efforts into AI models, informed by the platform’s avatar design and a collection of Roblox-specific 3D avatar models. Techniques under consideration include generating 3D avatars from 2D visuals, fortifying limited 3D data via 2D generative strategies, and employing a generative adversarial network (GAN) for 3D generation. Moreover, the ControlNet technology is being harnessed to superimpose preset poses, enhancing the multi-angle avatar visuals.
Following the creation of the 3D avatar structure, advanced 3D semantic segmentation, informed by avatar poses, is utilized. This procedure refines the initial 3D design, incorporating distinguishing facial elements, structures, and textures, ultimately metamorphosing the basic 3D form into an authentic Roblox avatar. Subsequently, users can employ a mesh-editing tool for further customization, aligning the avatar with their envisioned concept. Remarkably, Roblox AI allows this process to be completed within mere minutes, enabling immediate avatar integration into Roblox experiences.
Moderating voice communication
Roblox AI extends beyond mere content creation; it also serves as a robust mechanism to foster a varied, secure, and respectful community on a vast scale. With the introduction of novel voice features, such as voice chat, Roblox Connect, and newly announced APIs at RDC, moderating real-time spoken content emerges as a newfound challenge. The prevalent solution in the industry, Automatic Speech Recognition (ASR), functions by converting audio files into text, which is then scrutinized for potentially inappropriate content or specific keywords.
While effective for smaller-scale entities, deploying ASR for Roblox’s vast user base revealed its inefficiencies. Not only does it miss out on capturing nuances like tone and volume, but the sheer volume of daily conversations, spanning multiple languages, makes it cumbersome. Recognizing these challenges, Roblox AI pioneered a more streamlined procedure that identifies policy breaches directly from live audio.
This state-of-the-art voice-detection mechanism was developed internally. Roblox AI employed ASR to categorize its proprietary voice datasets and subsequently utilized the categorized data for system training. The training process commences with converting audio into transcripts, which are then assessed by Roblox’s text filter system, already adept at identifying Roblox-specific jargon and potential breaches. Consequently, Roblox now has a model proficient in real-time detection of policy violations directly from audio streams.
The Roblox AI system while adept at identifying explicit keywords, recognizes that policy breaches often depend on the broader context. A word that’s objectionable in one setting might be benign in another. To improve contextual understanding, Roblox integrated a transformer-based architecture, adept at sequence analysis. This framework preserves extended audio sequences, enabling the system to discern words, context, and intonations. The outcome is a system that classifies audio based on policy adherence, factoring in keywords, tone, sentiment, and overall context. Impressively, this innovative system requires significantly less computational power than traditional ASR, aligning with Roblox’s vision of expansive scalability.
