Google I/O 2024 - Gemini 1.5 Pro, Gemini app, A.I. in Google Search and more announced

At the annual developer conference, Google CEO Sundar Pichai announced that Gemini 1.5 Pro that was announced just two months ago, will now be available to the public via Gemini Advanced

In Short:

Sundar Pichai opens Google I/O 2024 discussing AI

Over 2 billion users and 1.5 million developers use Gemini

Gemini 1.5 Pro now available to all via Gemini Advanced

At the Google I/O 2024, Sundar Pichai opened the event talking about the biggest talking point for both its users and developers –– A.I. Sundar Pichai claims that over 2 billion users use Gemini, and more than 1.5 million developers use Gemini APIs. At the event, as expected, Sundar Pichai announced the expansion and deeper integration of AI within its apps and products. Pichai announced integration of Gemini A.I. into Google Photos. With the integration, you can ask Google Photos to simply, say, tell you what your license plate number is, or something even more specific. Gemini will look through your images and show you the answer along with relevant images.

The Google CEO also says A.I. overviews will start rolling out to all users in the USA this week, and it will eventually be rolled out to users in other countries as well.

Gemini 1.5 Pro now available for all

Sundar Pichai also announced that Gemini 1.5 Pro –– the latest version of its A.I. model –– will now be available to all users via Gemini Advanced. The version open to the public has a context window of 1 million tokens. Meanwhile, Google has now also bumped up Gemini 1.5 Pro to process 2 million tokens, but that will only be available for developers in a private preview. It has the "longest context window in the world". Users can uploading 30,000 lines of code of 1 hour long video on Gemini Advanced.

Gemini Advanced will also have a Data Analytics feature launching soon. It will allow users to upload charts and reports and the chatbot will be able to anylyse it for users. At the end of the year, the token on the platform will also be doubled to 2 million for all users.

Starting today, Gemini Advanced will also be expanding to 35 more languages.

Additionally, Gemini integration into Gmail was also demoed. Gemini will now be able to summarise emails, give highlights of Google Meet recordings, prepare responses, among other things. Gemini 1.5 Pro is now available in Workspace Labs.

Gemini app

Gemini app will now allow users to have voice conversations with the A.I. chatbot. The feature is called Live. It is very similar to the ChatGPT 4o demo we saw at the OpenAI Spring Update event yesterday. Google claims the responses in the app will be very natural, and you can interrupt it at any point. The company engineer says that the tool specialises in narrating and creating short stories.

Gemini app will also be able to plan and take actions on a user's behalf. Users will be able to plan trips and search for a restaurant all using teh Gemini app. Basically like using Google Search but via the Gemini app.

Gemini in Google Search

"Google Search is Generative AI at the scale of human curiosity, and it is our most exciting chapter yet," says Google CEO Sundar Pichai.

Google says A.I. Overview in Search is going to be more helpful now. Search will now apparently understand more complex searches with specific instructions. Google calls it multistep reasoning. Google demoed using Search to plan a holiday. Essentially, what we have been doing with ChatGPT, Google now wants people to directly search for that on Google Search. Results of Google Search will be collaborative and can be shared with other users.

Search will soon also have A.I.-organised search page which will be customisable and personalised for users.

Somewhat a next step to image search, Search will now allow video search. Essentially, users can share a video with a query in search and A.I. Overview will look up answer from the internet and provide users with solutions and results.

All these features will roll out in Google Search in the coming weeks.

Gemini for Workspace

Gemini in Google Meets is expanding to 68 languages. Gmail will now have a summarise email on top of the app. Users will also be able to search for queries and questions within Gmail using Gemini. Smart reply feature in Gmail has also been upgraded. It can now understand context and respond and offer suggestions accordingly.

Gemini in Gmail will also allow users to set workflows on a weekly or monthly basis to keep track of receipts or reports.

Similarly, in Sheets, Gemini can be used to analyse and summarise data with simple text prompts.

These capabilities will rollout to Lab users this month.

Google Audio Overview and A.I. Agent announced

Google also demoed an Audio Overview feature, which is specifically targeted towards students. The feature will essentially explain technical concepts in a conversational way, with simple examples like of basketball.

Sundar Pichai also demonstrated a new A.I. tool called Agent –– which is supposed to be still developing –– that is meant to organise and reason on your behalf. The tool is still a prototype, according to Sundar Pichai.

Gemini 1.5 Pro Flash

Google has also announced a new Gemini 1.5 Pro Flash. Flash is optimised for low latency tasks and is available in Google A.I. Studio. Similar to 1.5 Pro, 1 million context windows in Flash are available to all users, and for developers that context window is bumped to 2 million. Flash is essentially a lighter version but it is faster in its responses.

Gemini in Google Camera, Imagen 3, Veo and more

Similar to ChatGPT 4.0 A.I. assistant demo we saw yesterday, Google has also showed off A.I. integration in the Google Camera app. What was really cool was that the A.I. in the camera also could hold on to context. As per the demo, it can also remember what you showed it in the past. For instance, in the demo, the engineer shows the camera multiple desks, and asks it random question and then eventually says "where did you see my glasses last, and interestinly, it was able to tell where they exactly were in that room.

Google has also announced the latest Imagen 3, which it claims will understand text prompts more accurately. It will be available in A.I. Labs soon, but will likely be available only to developers initially.

Google Veo is the company's new Generative Video A.I. tool. Similar to OpenAI's Sora, it is a text to video A.I. tool. Veo is currently only available via Labs and for developers.

6th generation of TPUs: Trillium

Google has announced new 6th generation TPUs called Trillium which is able to deliber 4.7x improvement in computing performance over the previous generation. "Most efficient and performant TPU to date," says Sundar Pichai. The TPUs have been made in partnership with Nvidia. It will be available later this year to cloud consumers.

TPUs (Tensor Processing Units) are custom hardware accelerators developed by Google for machine learning tasks. Optimised for TensorFlow, they outperform CPUs and GPUs in deep learning computations. TPUs enhance performance and energy efficiency, pivotal in Google's infrastructure for AI applications like natural language processing and image recognition.