At Google I/O 2024, the search engine unveiled a new virtual assistant, Project Astra, based on the Gemini language model, that can analyze video, voice, and text to answer all questions.
Google Assistant’s hours seem to be numbered. Google has just announced, during the Google I/O conference dedicated to its software innovations, a new assistant called, for the moment, Project Astra. The latter is based, of course, on the linguistic model of the American brand, Gemini. The latter uses Google’s computing power to analyze text, voice, and images to get contextual answers to every question asked.
An assistant that always listens and sees what you see
Project Astra was not presented on stage, but through a video, shot in one breath and without editing, according to the Google teams. This technology is not yet a finished product, but a working project of Google DeepMind, the team that deals with artificial intelligence research.
In this excerpt, we can see a user launch the voice assistant, then open the camera of his smartphone. This way, Project Astra can see what the demonstrator sees to provide answers to all their questions.
In the examples staged, Project Astra was able to understand a piece of code caught on camera and give directions to improve it. It can also recognize objects, or make suggestions based on the elements in front of it and the questions asked by the user.
A research project that foreshadows Google’s ambitions in the coming years
Even better: Project Astra analyzes a range of data when the phone is moved from one place to another and in real time. In the same video, the user asks where their glasses are, and the AI is able to remind them of the exact place where they were left.
Project Astra is just a proof of concept and won’t be integrated into Android for several months or even years. It’s also very similar to GPT-4o, OpenAI’s latest language model unveiled on Monday, May 13, and which has the same features, with a voice that could be described as more natural.
Before then, users will be able to take advantage of Google’s advances in artificial intelligence with Gemini’s integration into all of Google’s services, including the search engine, Gmail, and Google Workspace office applications.
