Perception Encoder Audiovisual (PE-AV)

### TL;DR

Meta's Perception Encoder Audiovisual (PE-AV) is a state-of-the-art multimodal model that embeds audio, video, audio-video, and text into a joint embedding space. Trained using contrastive learning on approximately 100 million audio-video pairs with text captions, PE-AV enables powerful cross-modal retrieval and understanding across audio, video, and text modalities. ([marktechpost.com](https://www.marktechpost.com/2025/12/22/meta-ai-open-sourced-perception-encoder-audiovisual-pe-av-the-audiovisual-encoder-powering-sam-audio-and-large-scale-multimodal-retrieval/?utm_source=openai))

Key Insights & Metrics

Key Features

• Embeds audio, video, and text into a unified embedding space
• Trained on 100 million audio-video pairs with text captions
• Enables cross-modal retrieval and understanding
• Achieves state-of-the-art performance on multiple benchmarks
• Supports tasks like video retrieval from text and audio retrieval from video

→ Related Releases

Discussion