Google DeepMind has launched agentic video understanding, an active video analysis capability for the Gemini 3.7 Flash, 3.6 Flash, and 3.5 Flash-Lite models. According to the announcement on September 1, this feature reduces token consumption by up to 88%, cuts analysis costs by 66%, and improves accuracy by up to 7% compared to existing static processing methods.
The Problem with the Old Approach
Static video analysis operates by ingesting entire videos at a fixed frame rate (defaulting to 1 FPS, configurable via API). While this works for short clips, for long-form content (ranging from 10-minute tutorials to 90-minute lectures or multi-hour recordings), developers are forced to choose between steep token costs or downsampling techniques that discard critical details.
How Is the New Approach Different?
Agentic Video Understanding allows Gemini to take an active, goal-driven role in deciding what to watch, at what playback speed, and through which modality (frames, audio, or transcript). The model retrieves only the necessary segments and modalities rather than ingesting the entire video stream.
Previously, developers had to write this logic manually; now, Gemini can execute it autonomously through an agentic loop, calling internal tools to retrieve relevant video segments and significantly reducing development overhead.
Agentic Video Understanding Capabilities:
-
Sub-second moment retrieval: Accurately pinpoints fleeting state changes and shot cut boundaries that are easily missed at 1 FPS, opening the door to precision video editing automation.
-
Needle-in-a-haystack search across long video: Answers complex queries across multi-hour videos without burning through millions of tokens.
-
Anomaly detection: Resamples key time windows at higher frame rates to inspect fast-moving action and subtle visual details.
-
Action and object counting: Accurately tracks repetitive motions and distinct objects over time.
Among the three supported models, Gemini 3.7 Flash with agentic understanding delivers the best overall quality and an optimal balance of quality and cost—defining the Pareto frontier for accuracy per dollar among the models tested for video analysis.
How to Enable and Pricing
The feature is available today for uploaded videos and YouTube videos via the Gemini API in Google AI Studio and the Gemini Enterprise Agent Platform. Developers simply need to set the processing configuration to “agentic” in the API. Pricing follows standard Gemini API token rates with no additional feature fees.
Google also noted that it will roll out the capability to all Gemini app users on Flash and Flash-Lite models going forward, and in the coming months will upgrade YouTube’s Ask YouTube feature to provide better visual-grounded answers. Early testing partners—including Ponder, Revyl, Mosaic, and Resemble.AI—all reported noticeable performance gains.