Google Lens Upgrades with Real-Time Video Analysis

Google lens upgrades its visual search app, adding a new feature that enables near-real-time video analysis and answers to questions about your surroundings. Available to English-speaking Android and iOS users with the Google app installed, Lens now allows users to capture videos and ask questions about objects of interest in real time.

The feature uses Google’s Gemini model, an advanced AI technology, to interpret the video and provide answers. Lou Wang, director of product management for Lens, highlighted the feature’s capabilities: “Let’s say you want to learn more about some interesting fish. Lens will produce an overview explaining their behavior, alongside additional resources and helpful information.”

New Video and Shopping Features

To access the video analysis feature, users must sign up for Google’s Search Labs program and opt in to “AI Overviews and more.” Once activated, Lens allows users to capture videos and ask questions by simply holding the shutter button on their phone’s camera. The AI will identify relevant frames and generate answers through AI Overviews, summarizing web content in response to user queries.

In addition to video analysis, Google has enhanced Lens’ shopping capabilities. When users capture or upload photos of products, Lens will now display information such as price, deals, reviews, and stock availability. This feature is available for select product categories like electronics, toys, and beauty, and is supported in specific regions.

Enhanced AI Capabilities

Lens isn’t the only tool exploring real-time video analysis. Meta has previewed a similar feature for its AR glasses, Ray-Ban Meta, and OpenAI has teased the ability for its Advanced Voice Mode to analyze videos. However, Google seems to have taken the lead in releasing video functionality, even if Lens currently operates asynchronously rather than in real-time conversation mode.

With approximately 4 billion Lens searches related to shopping each month, Google is also capitalizing on this opportunity by integrating ads into product search results. This move will help users find relevant shopping deals while enhancing Google’s advertising revenue.

Expanding Visual Search

In addition to these upgrades, Google Lens has also introduced new image and text search functionalities, enabling users to search with both text and photos simultaneously. This feature is available to all English-speaking users, regardless of whether they are enrolled in Google Labs.