Streaming video agents can already watch and answer at the same time. But the moment the thing worth seeing has not surfaced on screen yet, or the user's question is vague, the model is squeezed between two bad options: keep waiting, or answer and hallucinate. A new benchmark called AskStream, published by researchers from the Institute of Automation at the Chinese Academy of Sciences and Beijing Academy of Artificial Intelligence among others (paper: https://arxiv.org/abs/2610.38678), puts that squeeze on the table and proposes a third way out. Active interaction should not only offer "wait" and "answer" — it needs a third action: ask.

From wait-and-answer to wait, ask, answer

The premise behind existing proactive video understanding is simple: the information needed to answer will eventually show up in the video. Models are therefore left to choose between waiting longer and answering now. Real scenarios break that premise constantly. The camera never pans to the object hidden behind the cup, or the user says "remind me when that person appears" without saying which person. Waiting adds no new information there; answering means making something up.

AskStream widens the action space to three: wait, ask, and answer, and it splits "uncertainty" along two axes. Video-side uncertainty covers missing key visual evidence, missing viewpoint information, missing individual concepts, and missing personalized action concepts. User-side uncertainty covers ambiguous questions, questions with false premises, missing user preferences, and conflicting preferences. That yields four task categories and eight subtasks.

438 videos and 2,714 action windows

The benchmark is not huge, but the annotation is heavy. It contains 438 videos and 1,401 questions across nine domains, with clip durations ranging from 16.72 seconds to 118 minutes and an average of 3.2 interaction turns per video. Rather than marking single timestamps, the authors mark valid action windows: wait, ask, and answer windows accumulate to 33.92, 30.47, and 12.00 hours of effective duration respectively, across 2,714 manually annotated frame-level timestamps in total. Annotation reliability was checked too — 160 sampled instances were re-annotated independently by two annotators, reaching a mean temporal intersection-over-union of 0.962 for ASK windows and 0.971 for ANSWER windows.

Evaluation runs in both a multiple-choice and an open-ended setting. In the open-ended setting Gemini-3.8 Flash serves as the judge; the authors sampled 300 turns for human review and reported 93.50% overall agreement between the Gemini judge and human consensus, with no clear self-preference (93.75% on Gemini-3.8 Flash's own outputs versus 93.27% on outputs from other models).

52.18% offline, 7.22% streaming

The scores are the sharpest part. In the paper's multiple-choice evaluation, the highest offline closed-source Ask-F1 belongs to Gemini-3.8 Flash at 52.18%, with the asking sub-score at 44.63% and the answering sub-score at 62.80%. GPT-5.6-Sol reaches 20.50% and Claude Sonnet 5 reaches 22.48%. Offline open-source models sit lower still: MiMo-V2.5 at 14.66%, Kimi K2.6 at 16.52%, Qwen-3.8 Flash at 17.60%, and DeepSeek-V4.1-Flash at 19.69%.

The awkward part is what happens to models built specifically for streaming interaction. In the paper's evaluation MOSS-VL-Realtime, MMDuet2, and VideoLLM-online score zero on most subtasks; MiniCPM-o 4.5 reaches 0.12, TimeChat-Online 4.18, JoyAI-VL-Interaction 4.91, and AURA — the highest-scoring of the group — only 7.22%. Switch to the open-ended setting and the picture drops further: Gemini-3.8 Flash falls to 29.19% and AURA to 4.01%. The paper's read is blunt: these models are not bad at being proactive in general, they are bad at asking.

The real bottleneck is when to ask

Two sets of ablations separate the two dimensions of asking.

One ablation is an oracle experiment. On the subtask about missing key visual evidence, telling the model the correct answer to "when should I ask" alone lifts Ask-F1 from 37.17% to 58.30%, a gain of 21.13 points. Supplying both "when to ask" and "what to ask" raises overall Ask-F1 by 43.83% per the paper. The missing piece is timing, not wording.

The second ablation varies the ask budget. Letting the model ask more often — sweeping the budget from 1 up to 16 — lifts the score but saturates around 52.35%. More opportunities to ask do not substitute for knowing that you should ask.

There is also a detail worth keeping: before any of this, the authors ran a blur experiment to check whether asking is worth anything at all. With a blurred frame the model answers correctly 42.3% of the time. After the model requests and receives the right evidence, accuracy rises to 62.3%, close to the 63.1% it reaches with a fully clear input. Feed in mismatched evidence instead and the benefit largely disappears. What helps is not seeing more frames — it is obtaining information that matches the current question.

Memory structure gets its own accounting. With StreamHarness, a hierarchical memory framework that keeps the most recent 10 frames at 1 fps, summarizes every 10 frames into one mid-term memory, consolidates every four mid-term memories into one long-term block, and retains up to eight long-term blocks, Ask-F1 is 52.18%. Dropping to a sliding window that keeps only the last 10 frames gives 41.70%, uniform sampling of 128 frames gives 26.72%, and 64 frames gives 16.99%.

What the benchmark is pointing at

AskStream deliberately makes the task hard: every model is evaluated under conditions where it either cannot see clearly or cannot tell what is being asked, and the model gets one action opportunity per second with a tolerance of plus or minus one second. The paper also lists four limitations of its own — at most one ASK action per turn, coverage limited to streaming video understanding without embodied agents, every interaction initiated by an explicit user query, and real uncertainty that plays out over longer and messier time horizons.

But the design points at a stage the industry keeps skipping. Almost no model handles "knowing when to ask" well: even a strong offline model clears only a bit more than half of the bar, and the streaming models built for real-time interaction mostly score zero.

The next phase of streaming video looks less about seeing more clearly and more about knowing that you cannot see clearly. AskStream turns that ability into a 52.18% threshold, and mainstream streaming models are still far below it. So the next time a leaderboard like this refreshes, the number worth watching may not be whose score climbed — it may be which model dares to open its mouth and ask at the right moment.