[Gemini Update] Project Astra: Experience AI Agents in the Real World 🥽

First introduced at Google I/O, Project Astra is an AI agent that utilizes multimodal understanding in the real world, aiming to be a smarter and more useful AI assistant through interaction with users. Since then, feedback provided by a trusted group of testers using it on Android devices has played a crucial role in improving Project Astra’s performance and ethical design. The latest version developed based on Gemini 2.0 offers enhanced features in several aspects.

→ Try Stream realtime on your mobile!

Access the link and press the Show Gemini button

In the video below, you can see how Gemini interacts with the real world through the camera.
Please be sure to watch the video 👀

Key Improvements

1. Better Conversational Ability

Project Astra now supports multiple languages and mixed languages, and can better understand various accents and uncommon words. This provides a more natural and flexible conversational experience.

2. New Tool Integration

Based on Gemini 2.0, Project Astra is integrated with the following Google tools to support users’ daily lives:

Google Search: Quickly search for necessary information.
Google Lens: Analyze photos and text.
Google Maps: Provide location-based services.

3. Improved Memory

The memory function, designed to be user-controlled, has also been improved. Project Astra retains memory for up to 10-minute sessions, better remembering past conversations to provide personalized interactions.

4. Reduced Latency

With new streaming capabilities and native audio understanding, we’ve achieved fast response times similar to talking to a person. This allows users to experience more immediate and natural conversations.

Expanded Applicability

We are working to apply these features to Google products like the Gemini app, an AI assistant, and other form factors like glasses. And we have begun expanding our trusted tester program to more people, including a small group to test Project Astra with “prototype glasses” soon.

Ethical Design and Future

Feedback from the tester group plays a crucial role in ensuring that Project Astra is designed safely and ethically. Based on this feedback, Google is continuously working to develop an AI assistant that users can trust and feel secure using.

Project Astra, through its human-centered approach as well as its technological innovation, shows how AI can be a better tool in our daily lives. We look forward to more developments through this innovative journey in the future.