See what changes.
Preserve fine-grained actions and short-lived transitions instead of relying on sparse still frames.
Research release Generalist video MLLM
VideoChat3
A generalist video MLLM built for fine-grained motion, long-form reasoning, temporal grounding, and live streaming.
Token efficient16× spatiotemporal compression
Fully openModels · code · data · training recipe
ADAPTIVE LIVE PERCEPTION05:40 / 06:00
Low↑ High↓ Low
01Silencemonitor efficiently
02Standbylook closer
03Responsespeak with evidence
Standby alone switches the next window to high resolution; Response returns to low resolution.
4Bparameters
16×spatiotemporal compression
3Mcurated instruction samples
20.4slatency at 2,048 frames
THE PREMISE
One model. Every tempo.
Most video models are optimized for one regime. VideoChat3 treats video as a continuous temporal signal—from a subtle hand movement, to an hour-long story, to the moment a live assistant should respond.
Remember what matters.
Connect evidence across minutes or hours, retrieve events, and ground answers to precise moments.
Know when to speak.
Observe causally, stay silent without enough evidence, and respond when the answer becomes available.
ARCHITECTURE / EFFICIENCY
Compress after understanding.
VideoChat3 models local space and time inside the visual tokenizer—before visual tokens reach the language model. Redundancy is removed early, while motion cues stay intact.
01
INFLATED 3D VISION TRANSFORMER
I3D‑ViT
Groups four neighboring frames, performs native-resolution spatiotemporal attention, then pools in time.
- 4× temporal compression
- 2×2 spatial pixel shuffle
- 16× combined compression
02
ADAPTIVE FRAME RESOLUTION
Attention where it counts.
Streaming windows start at low resolution. A Standby state raises the next window’s pixel budget to inspect emerging evidence.
Silence · low resolutionStandby · high resolution
DATA / EFFECTIVENESS
Three datasets. One continuum.
The data recipe moves from reliable academic labels to long-range evidence and causal response timing. Every stage is designed to make supervision denser without losing its grounding in the video.
Academic2M
Rewrites sparse labels into evidence-grounded answers, then filters for consistency with the original annotation.
LV116K
Builds timestamped evidence ledgers for temporal grounding, event timelines, and cross-segment question answering.
OL617K
Turns offline video QA into causal streams with explicit Silence, Standby, and Response supervision.
SilenceStandbyResponse
RESULTS
Small model. Long reach.
At 4B parameters, VideoChat3 improves on 18 of 19 directly comparable offline metrics and 10 of 11 streaming metrics against Qwen3‑VL‑4B under the paper’s evaluation settings.
+14.9 over Qwen3‑VL‑4B
+27.4 over Qwen3‑VL‑4B
2,048 INPUT FRAMES · NVIDIA H200
Less work for the language model.
I3D‑ViT shifts computation into a linear vision stage and halves the visual tokens that enter the quadratic language stage.
Qwen3‑VL44.45 seconds
VideoChat320.41 seconds
54% lower latency · 100,352 visual tokens · 80.8 GB
Results and efficiency figures are reported in the VideoChat3 paper. Benchmark protocols differ across suites.
QUALITATIVE DEMOS
See time become evidence.
From retrieving a detail deep inside a long video to waiting for the right moment in a live stream, these examples show the same model working across different temporal scales.
Observe, wait, then respond.
Retrieve details across extended context.
Map language to precise boundaries.
FULLY OPEN
Fully open. Build on it.
VideoChat3 releases the pieces that turn a result into a research foundation: model weights, training code, training strategy, complete training datasets, and the data construction pipeline.