Alibaba has released Qwen3.8-Omni-Flash, its latest omnimodal artificial intelligence model designed for low-latency visual, audio, and language tasks. The release moves beyond passive perception toward active task execution. Rather than only transcribing audio or identifying visual objects, the system plans multi-step tasks, invokes external tools, and generates finished media assets.
The model features a native 1-million-token context window that processes text, images, audio, and video simultaneously. According to the Qwen official announcement, text benchmark scores remain on par with equivalent text-only models, while multi-modal evaluation scores improve by an average of 25% over Qwen3.5-Omni-Plus across 29 separate benchmarks.
Alibaba launched Qwen3.8-Omni-Flash with native omnimodal capabilities and a 1M-token context window.
Architecture and cost efficiency
Qwen3.8-Omni-Flash uses a Mixture-of-Experts architecture with hybrid attention layers based on the Qwen3.8-Flash-Next codebase. It incorporates a native thinking mode, which allows the model to run step-by-step reasoning across visual and auditory inputs before delivering an answer. This internal chain of thought helps resolve complex spatial relationships, transcribe noisy multi-speaker recordings, and coordinate multi-stage workflows.
Processing long-form audio and video traditionally creates steep inference costs. In this release, the API price per hour of audio input decreased by more than 98%, while the price per hour of audio-visual input fell by more than 93% compared to previous generations. The lower operational cost makes continuous multi-modal monitoring and long-form analysis practical for production deployments.
Performance and input pricing comparison across omnimodal model generations.
Agentic video understanding and token reduction
Standard multi-modal models read video files by ingesting every frame in sequence from start to finish. When answering questions about a two-hour recording, this approach consumes hundreds of thousands of tokens, even if the relevant event lasts only a few seconds. Qwen3.8-Omni-Flash introduces an agentic evidence gathering loop: the model reviews the user query, determines which timestamps require closer inspection, and zooms into specific visual segments.
This selective inspection reduces token consumption while raising accuracy. On the OmniVideoBench evaluation, agentic evidence gathering reduced token consumption per query from 145,736 tokens down to 79,117 tokens, a drop of 45.7%. At the same time, response accuracy rose from 63.4% to 67.8%.
Evaluation Metric
Static Understanding
Agentic Understanding
Accuracy (OmniVideoBench)
63.4%
67.8%
Tokens per query
145,736
79,117 (-45.7%)
Comparison of static frame ingestion versus agentic coarse-to-fine evidence gathering.
Demonstration of agentic long-form video question answering and targeted evidence retrieval.
Long-form audio-visual workflows
The model connects media comprehension with tool execution to handle complex media production tasks without manual handoffs between separate tools:
Music2MV: Analyzes song tempo, mood, melodic transitions, and vocals. It outputs line-level lyrics with millisecond timestamps to align character motion, camera angles, and visual pacing with the audio track.
Video translation and dubbing: Performs speaker diarization, translates conversational dialogue, clones character voices, and matches speech length to visual scene cuts in a unified pass.
Film commentary generation: Ingests feature-length films lasting two to three hours. The model identifies plot arcs, drafts narration scripts, interleaves original dialogue with voiceovers, and balances background audio.
Meeting diarization and action tracking: Transcribes up to one hour of multi-party audio-visual meetings. On the AliMeeting benchmark, the Diarization Error Rate (DER) dropped from 88.11% to 3.35%, while the concatenated word error rate (cpWER) dropped from 89.61% to 17.18%.
Overview of automated production pipelines for video editing, localization, and commentary.
Autonomous model optimization
In an experimental trial, Alibaba engineers assigned Qwen3.8-Omni-Flash to improve Sichuan dialect speech recognition for Qwen2.5-Omni-3B within a 12-hour deadline. The agent established a baseline on the WenetSpeech-Chuan test set, identified phonetic recognition errors, and generated 3,413 targeted training samples. Through four successive fine-tuning rounds, the agent evaluated progress and rolled back ineffective changes. The character error rate dropped from 25.79% to 15.30%, demonstrating how large multi-modal systems can guide the training of smaller, domain-specific models.
Information distillation and skill creation
Long video tutorials contain valuable practical knowledge that is difficult to index and search. Qwen3.8-Omni-Flash addresses this with two open-source modules:
Video2Note: Automatically extracts instructional steps from technical videos, captures representative screen frames, and compiles structured PDF reference documents.
Omni Skill Creator: Converts screen recordings and software demonstrations into standardized agent skill files, formalizing software operating procedures into reusable instructions.
Benchmark ratings across modalities
Qwen3.8-Omni-Flash underwent evaluation across agentic tool usage, audio-visual comprehension, text reasoning, and software engineering. The results indicate strong parity with proprietary frontier models such as Gemini 3.8 Flash and Claude Opus 4.6.
Performance across agentic tool usage, video reasoning, and multi-speaker audio benchmarks.
Code and reasoning benchmarks
The omnimodal architecture maintains high performance on standard text, math, and software engineering benchmarks without suffering from multi-modal alignment degradation:
Software engineering and analytical reasoning benchmarks.
Real-time interaction and latency benchmarks
The Realtime API provides low-latency bidirectional voice and video communication over WebSockets. Speech recognition covers 74 languages and 39 Chinese dialects, while voice generation supports 29 languages and 7 dialects. Across short audio inputs, time to first token measures under 600 milliseconds, and audio packet generation starts within one second.
Input Scenario
Text Output TPS
Time to First Token
Time to First Audio Packet
Audio Real-Time Factor (RTF)
Audio (6 seconds)
84.87 tokens/s
591 ms
978 ms
0.1538
Audio (12 seconds)
83.18 tokens/s
604 ms
982 ms
0.1537
Audio-Visual (6 seconds)
84.89 tokens/s
837 ms
1,214 ms
0.1524
Audio-Visual (12 seconds)
84.33 tokens/s
911 ms
1,268 ms
0.1527
Realtime API latency and throughput across audio and audio-visual input durations.
Developer ecosystem and open-source tooling
To support production deployments, Alibaba released two open-source companion frameworks:
Qwen-Live Harness: A runtime client that manages long-horizon conversation context, delegates secondary tasks, and maintains stateful memory during live audio-visual streams.
Qwen-MM-Plugins: A plugin suite providing specialized capabilities such as document parsing, visual object localization, video editing, and agent skill extraction.
Installation and command-line execution for Qwen-MM-Plugins.
