Trinity Large
### TL;DR
Trinity Large is a 400-billion parameter sparse Mixture-of-Experts (MoE) model developed by Arcee AI. It utilizes 256 experts with 4 experts active per token, resulting in a sparsity ratio of 1.56%. The model was trained on 17 trillion tokens using 2,048 Nvidia B300 GPUs over a period of 33 days.
Key Insights & Metrics
Key Features
- 400-billion parameter sparse MoE model
- 256 experts with 4 experts active per token
- Sparsity ratio of 1.56%
- Trained on 17 trillion tokens
- Utilized 2,048 Nvidia B300 GPUs over 33 days
→ Related Releases
Protenix
Protenix is an open-source, trainable PyTorch implementation of AlphaFold 3, designed for high-accuracy biomolecular structure prediction. It aims to advance accessible and extensible research tools for the computational biology community. ([github.com](https://github.com/bytedance/Protenix?utm_source=openai))
Z-Image
Z-Image is an efficient image generation foundation model developed by Tongyi-MAI, designed to produce high-quality, diverse, and stylistically versatile images. It serves as a robust base for creators, researchers, and developers seeking advanced image generation capabilities.
Discussion
Sign in to leave a review
Reviews
No reviews yet. Be the first to review!