Qwen3.8-Omni-Flash: AI Model for Audio, Video and Autonomous Agents
Alibaba Cloud’s Qwen team has officially released Qwen3.8-Omni-Flash, marking a major milestone in the evolution of native omnimodal artificial intelligence. The release shifts the focus of multimodal models from passive content understanding to active task planning, automated tool calling, and end-to-end creative execution in real-world productivity scenarios. Alongside this functional leap, the release delivers a dramatic reduction in operational costs, slashing API pricing by more than 98% for audio inputs and over 93% for audio-visual inputs, all while supporting a massive 1-million-token context window.
Performance Benchmarks and Dramatic API Cost Reductions
According to technical documentation released by the Qwen team, Qwen3.8-Omni-Flash achieves an average score improvement of more than 25% across 29 specialized benchmark evaluations compared to its predecessor, Qwen3.5-Omni-Plus. Crucially, this multimodal expansion comes with zero compromise on text capabilities, matching the performance of dedicated text-only models of equivalent size.
On the economic front, the model drastically lowers the barrier to deploying audio and video agents in production. The hourly API cost for processing audio inputs drops by more than 98%, while audio-visual input processing sees a reduction exceeding 93%. This pricing structure makes running continuous, high-volume multimodal agents commercially viable for businesses and developers alike.
The model also achieved substantial gains across complex, long-horizon agent benchmarks:
- WildClawBench-MM (Multimodal Tool Use): Scored 71.0, representing a 36.5-point jump over Qwen3.5-Omni-Plus.
- AgenticVBench (Video Agent Tasks): Scored 36.8, an improvement of 22.3 points.
- UniClawBench: Delivered a strong score of 69.6 in multimodal agent workflows.
Benchmark evaluations place Qwen3.8-Omni-Flash’s audio-visual reasoning on par with Google’s Gemini 3.8 Flash, while its overall audio capabilities surpass Gemini 3.8 Flash. These advances highlight a fundamental shift: audio and video are no longer treated simply as sensory inputs, but as primary media through which AI agents perceive their environment, reason, and execute end-to-end tasks.
Agentic Video Understanding: Slashing Token Consumption by 45.7%
Conventional approaches to long-form video processing force models to ingest an entire recording frame by frame, even when the user’s query pertains to just a few seconds of footage. To solve this inefficiency, Qwen3.8-Omni-Flash introduces Agentic Long-Form Audio-Visual Understanding. Instead of processing every single frame, the agent starts directly from the user’s query and autonomously navigates the footage, gathering evidence through iterative coarse-to-fine inspection rounds.
OmniVideoBench Performance Highlights:
- Reasoning Accuracy: Increased from 63.4% in static processing to 67.8% in agentic mode.
- Token Consumption per Query: Dropped from 145,736 tokens down to 79,117 tokens.
- Compute Efficiency Gain: Achieved an approximate 45.7% reduction in token consumption per query.
Complementing this capability is Controllable Audio-Visual Captioning. Users can specify the exact subject, time window, level of detail, and desired output format. Whether developers require a high-level plot summary, precise character movement tracking, or an in-depth breakdown of lighting, camera angles, and sound effects, the model outputs structured data conforming to strict JSON Schema requirements.
Long Meeting Intelligence: From Transcription to Code Execution
Multi-participant meetings represent one of the most challenging enterprise audio-visual scenarios due to overlapping dialogue, shifting topics, and complex speaker dynamics. Qwen3.8-Omni-Flash natively supports up to one hour of continuous audio-visual input, performing speaker diarization, speech transcription, and visual identity alignment end-to-end.
On the rigorous AliMeeting benchmark, the model reduced the Diarization Error Rate (DER) from 88.11% down to 3.35%, while the concatenated minimum-permutation Word Error Rate (cpWER) plummeted from 89.61% to 17.18%.
Beyond generating meeting minutes, the model utilizes visual context to resolve ambiguous spoken references, maps organizational relationships among participants, extracts action items, and evaluates project risks. When connected to external software harnesses, it can immediately draft follow-up emails, organize task boards, and even begin writing code based on meeting decisions.
Multimodal Deep Research for Complex Problem Solving
When users analyze video content, the answers they need frequently require contextual knowledge found outside the footage itself. Qwen3.8-Omni-Flash combines the user’s query with video evidence to identify key research questions, search the broader web across images, videos, and technical documents, and synthesize a richly illustrated research report.
In a practical demonstration, when a user faced color fringing issues after cutting out hair in Adobe Photoshop, the model parsed the video tutorial, investigated the mathematical principles behind Multiply and Screen blending modes, evaluated alternative edge-refinement techniques across the web, and provided a comprehensive troubleshooting guide tailored to the user’s workflow.
Automated Audio-Visual Production Workflows
Qwen3.8-Omni-Flash transitions audio-visual agents into full creative production, autonomously planning timelines, invoking external creative tools, and delivering finished media assets:
- Music Video Creation (Music2MV): Analyzes the tempo, musical structure, emotional tone, and instrumental shifts of an audio track to plan matching characters, scenes, and camera shots, aligning line-level lyrics and subtitles with visual cues.
- Short Drama Localization: Replaces fragmented workflows with an end-to-end agent. The model performs speaker-aware dialogue recognition, natural conversational translation, character voice cloning, audio dubbing, sound remixing, and automated quality control.
- Long-Form Film Commentary: Ingests full-length movies lasting two to three hours, extracts pivotal narrative arcs, drafts engaging commentary scripts, balances narration with original audio and background scores, and renders a fully edited video essay.
Model Self-Optimization: Large Models Training Smaller Models
Deploying specialized AI at scale often requires compact, cost-efficient models fine-tuned for specific domain requirements. In a pioneering self-development experiment, the Qwen team tasked Qwen3.8-Omni-Flash with enhancing the Sichuan dialect speech recognition capability of the smaller Qwen2.5-Omni-3B model within a strict 12-hour window.
Autonomous Model Fine-Tuning Results:
- Character Error Rate (CER): Decreased from 25.79% to 15.30% on the WenetSpeech-Chuan evaluation set.
- Relative Error Reduction: Achieved an approximate 40.7% relative improvement.
- Synthetic Data Generation: Autonomously generated and validated 3,413 targeted training samples.
- Autonomous Iteration: Selected evaluation criteria, diagnosed failure points by listening to audio samples, executed four iterative training rounds, and rolled back unsuccessful experiments without human intervention.
This experiment demonstrates a viable path forward for enterprise AI: large multimodal models acting as autonomous research engineers to optimize lightweight edge models for specialized business needs.
Knowledge Distillation: Video2Note and Omni Skill Creator
To convert unstructured video knowledge into durable digital assets, the team open-sourced dedicated tools within the Qwen-MM-Plugins ecosystem:
- Video2Note: Leverages joint audio-visual understanding to parse complex instructional videos, identify key procedural steps, capture representative frames, and compile structured, publication-ready PDF notes.
- Omni Skill Creator: Watches screen recordings and expert demonstrations to extract Standard Operating Procedures (SOPs), converting specialist knowledge into executable
Skill.mdfiles that can be shared across agent frameworks.
Qwen-MM-Plugins natively integrates with leading agent harnesses, including Codex, Claude Code, Qwen Code, Gemini CLI, and OpenClaw.
Real-Time Interaction and Omnimodal Spatial Audio Perception
For low-latency interactive applications, the team introduced Qwen3.8-Omni-Flash-Realtime, supporting streaming bidirectional communication via WebSocket and WebRTC protocols.
Observed Realtime API Latency in Production:
- Audio Only (6s Input): 591.26 ms Time to First Token (TTFT); 978.36 ms Time to First Audio Packet.
- Audio-Visual (6s Input): 837.96 ms Time to First Token; 1214.73 ms Time to First Audio Packet.
- Audio-Visual (20s Input): 981.01 ms Time to First Token; 1350.49 ms Time to First Audio Packet.
The model is also the first omnimodal system to support Spatial Audio Perception. By correlating binaural spatial sound with visual feeds, it determines the precise direction and distance of sound sources while mapping physical obstacles. When given commands like “come over here” or “check what made that noise,” it isolates the target sound from background noise, localizes the source in 3D space, and invokes navigation tools to reach the target.
Additionally, the Realtime engine supports conversational language tutoring with accent-aware pronunciation correction, alongside dynamic injection of enterprise knowledge and custom customer service personas.
Language Support and Developer Integration
Qwen3.8-Omni-Flash offers extensive global language coverage, supporting speech recognition across 74 languages and 39 Chinese regional dialects, along with speech generation in 29 languages (including English, Arabic, Spanish, French, German, Mandarin, and Japanese) and 7 Chinese dialects.
Configurable Reasoning Effort Parameters (reasoning_effort):
xhigh(Default): Optimized for deep reasoning and complex software engineering tasks.medium: Balances analytical accuracy with rapid response times.low: Streamlined for speed and maximum cost efficiency.
The API is fully compatible with OpenAI standards (Chat Completions and Responses APIs), with preserve_thinking enabled by default. Developers can also deploy the open-source Qwen-Live Harness runtime for turnkey management of long-term memory, context windows, and real-time task delegation.
With Qwen3.8-Omni-Flash, Alibaba Cloud sets a new benchmark for multimodal AI-bridging the gap between understanding multi-sensory data and taking autonomous, cost-effective action in production environments.
