Opening. This role is part of the mid-training team at xAI, aiming to provide an omni model that can understand the universe through text, image, video, and audio.
What you'll do
The mid-training team at xAI aims to provide an omni model that can understand the universe through text, image, video, and audio. To accomplish this, we are looking for expert engineers in multimodal mid-training data.
- Scale synthetic coding data to trillions of tokens with large-scale docker verification.
- Distill the intelligence of flagship models into flash models through synthetic data generation.
- Optimize mid-training data mixtures to boost the ceiling for RL.
- Engineer long-context data recipes.
- Develop robust and diverse evaluation for mid-training checkpoints.
What you need
- Expertise in ML and large model scaling, with familiarity across all kinds of scaling laws.
- Strong ability to design ML experiments.
- Familiarity with state-of-the-art techniques for curating AI training data for text, image, audio, and video modalities.
- Strong engineering abilities in Spark, Ray, and other frameworks for large-scale data processing.
