SLAI T-Rex Goes Fully Open-Source: A World First for Trillion-Parameter MoE Post-Training on Domestic Computing Infrastructure
Introduction
On July 31, Shenzhen Loop Area Institute (SLAI) released the SLAI T-Rex technical report, laying out the technical approach and optimization methods for full-parameter post-training of trillion-parameter MoE models from the DeepSeek-V4 family on the Ascend SuperPOD.
The Institute is now releasing SLAI T-Rex as a fully open-source project, the first of its kind worldwide. It provides the complete engineering stack for full-parameter post-training of trillion-parameter MoE models on domestic computing infrastructure.
Most open-source releases share model weights, selected data samples, or training methods, which is enough for developers to use a model and grasp its design ideas. Reproducing and then improving the training of a trillion-parameter MoE model, however, is another matter. It requires knowing how the environment is built, how parallelism strategies are configured, how computation and communication are coordinated, how memory is managed, and how key operators are adapted to the hardware. These engineering details directly shape training efficiency and stability.
The full-stack release of SLAI T-Rex opens the entire training pipeline, from environment preparation and data construction to training execution and performance evaluation. Developers can inspect the actual implementations at every stage, see how the optimizations reinforce one another, and validate and improve them against their own hardware and task requirements.
These open engineering resources give more teams a foundation for post-training trillion-parameter models. They also provide a reusable reference for framework adaptation, operator optimization, and training methodology research on domestic computing infrastructure, sparing teams the cost of building from scratch and repeated trial and error.
01 Seven Categories of Open Resources and the Full Reproduction Workflow
The SLAI² Training Platform is releasing seven categories of resources:
Training images and environment dependencies
Training framework, training code, and operator optimization code
Training datasets and data processing pipelines
Training configurations and key parameters
Model weights
Evaluation sets and evaluation methods
End-to-end operation and reproduction guidance
With these in hand, developers and researchers can follow the guide through environment deployment, data preparation, task configuration, distributed training, weight generation, and performance evaluation, and then vary parallelism strategies, operator implementations, and system configurations to compare how each choice affects training efficiency and stability.
02 Core Open-Source Release 1: Training Data Construction and Validation Methods
Reproducing a model's performance requires both the corresponding training data and the processing pipeline. Along with SLAI T-Rex, the training data and construction methodology are also open-sourced, with sample quality guaranteed through data cleaning, independent review, and code execution validation.
Figure 1: AI-Assisted Data Quality Assurance Workflow
Candidate samples are first cleaned and repaired by the cleaner, then reviewed by an independent reviewer for logical and factual consistency, and finally put through automated code execution validation to confirm that they meet the required criteria.
Developers can inspect each stage of this workflow and leverage the released training configurations and evaluation methods to verify the impact of data adjustments on model performance.
03 Core Open-Source Release 2: The Training Engineering Behind Achieving an MFU of 38.57%
Further reproducing the training MFU requires the corresponding training images, environment, framework, code, and key operator optimizations—all of which are now fully open-sourced in this SLAI T-Rex release. Full-parameter post-training of a trillion-parameter MoE model means managing enormous model states while contending with communication overhead and computation scheduling. SLAI T-Rex integrates the DeepSeek-V4 architecture with the hardware characteristics of the Ascend SuperPOD to optimize parallel scheduling and key operators.
Figure 2: SLAI T-Rex Full-Stack Training Optimization Framework
Open-sourcing parallelism and scheduling optimizations: Tailored to the model architecture and SuperPOD interconnect characteristics, this release integrates data, tensor, pipeline, and expert parallelism, overlapping computation with communication to minimize stalls and pipeline bubbles.
Open-sourcing key operator optimizations: For modules such as sparse attention, mHC, limited SwiGLU, and RoPE, operators are rewritten in AscendC and further improved through operator fusion and data-flow optimization to cut redundant data movement and scheduling overhead. Before-and-after tests on typical modules showed an 8.86-fold speedup for the SwiGLU forward pass, 2.74-fold for mHC post, and 3.54-fold for the RoPE backward pass.
Figure 3: Comparison of Execution Time for Typical Modules Before and After Optimization
With these optimizations in place, the stable, full-parameter post-training of the trillion-parameter DeepSeek-V4 MoE model was successfully completed on the Ascend SuperPOD, achieving a training MFU (model FLOPs utilization) of 38.57%. The underlying code and configurations are released alongside this open-source update, enabling developers and researchers to reproduce the results, analyze performance, and drive further optimizations.
04 An Invitation to Reproduce, Validate, and Collaborate
With this release, the Institute hopes to turn the engineering experience accumulated through SLAI T-Rex into a foundation the research community can build on, supporting more teams to pursue full-parameter post-training of trillion-parameter MoE models on domestic computing infrastructure.
Universities, research institutions, developers, and industry partners are all welcome to reproduce and validate the results, report issues, share test outcomes, and contribute optimizations. SLAI looks forward to sustaining collaboration with the community on training efficiency, system stability, data quality, and domain capabilities, jointly refining the underlying engineering and methodologies to support future research and applications.
Related Links
Project Page：
https://slai-aitp.github.io/SLAI-T-Rex/
Technical Report：
https://arxiv.org/abs/2607.20145
Hugging Face Model (Pro)：
https://huggingface.co/SLAI-AITP/SLAI-T-Rex-Pro
Hugging Face Model (Flash)：
https://huggingface.co/SLAI-AITP/SLAI-T-Rex-Flash
GitCode Repository：
https://gitcode.com/SLAI/SLAI-T-Rex
GitHub Repository：
