NVIDIA Cosmos 3

### TL;DR

NVIDIA Cosmos 3 is a frontier open-source physical AI foundation model that unifies physical reasoning, world generation, and action generation in a single model using a Mixture-of-Transformers (MoT) architecture. It combines a Reasoner tower (VLM for understanding) and a Generator tower (diffusion-based video/action output), eliminating the need for multiple separate models. Available in two sizes — Cosmos 3 Nano (8B) and Cosmos 3 Super (32B) — with fully open model weights, training scripts, deployment tools, and six synthetic datasets on Hugging Face.

Key Insights & Metrics

Key Features

• Unified Mixture-of-Transformers (MoT) architecture — Cosmos 3 combines a Reasoner tower (autoregressive VLM that interprets images, video, and text to understand motion and physical context) with a Generator tower (diffusion-based process for physics-aware video and action output); the reasoner can run independently, while the generator activates both towers — eliminating multi-model orchestration and simplifying physical AI development pipelines
• Two model sizes for every deployment scenario — Cosmos 3 Nano (8B parameters) is optimized for workstation-grade inference on NVIDIA RTX PRO 6000 for real-time robotics; Cosmos 3 Super (32B parameters) targets datacenter Hopper/Blackwell GPUs for large-scale synthetic data generation; both available as NVIDIA NIM microservices for production-ready deployment without manual infra tuning
• Open-source SOTA across physical AI benchmarks — leads on PAIBench-G, R-Bench, Physics-IQ, RoboLab, VANTAGE-Bench (reasoning), and Artificial Analysis Text-to-Image and Image-to-Video leaderboards; ships with fully open training recipes (SFT + action post-training), six synthetic datasets covering robotics/AV/warehouses/physics/human motion, and the new Cosmos HUE evaluation framework for rigorous video generation quality assessment

Discussion