(Credit: Google)
Google is bringing AI-powered assistance to its entire lineup of smart home devices, including smart speakers and security cameras that debuted roughly a decade ago and a handful of new devices that the company will roll out over the coming months.
With early access already underway, Google’s conversational AI, Gemini, will replace the Google Assistant in the company’s smart speakers and smart displays. Indoor cameras, outdoor cameras, and video doorbells will utilize AI for semantic scene understanding, enabling more descriptive and specific notifications and making recordings easier to locate. The Google Home app is also getting a redesign to incorporate these changes, with the overall goal of making your smart home more proactive and intuitive to use.
Google announced the changes at a smart home-focused event in New York, and we got a chance to see some of these new features in action during a controlled demo beforehand. Here are the details of Google’s Gemini-powered smart home.
Gemini In; Google Assistant Out
The biggest change to Google’s smart home comes in the form of a new voice assistant. The conversational and generative AI assistant Gemini will replace the company’s long-standing digital helper, called Google Assistant, in Google’s smart speakers and smart displays. Like the rest of these changes, Google is making this update available to even its oldest devices, including the original Google Home smart speaker from 2016 and its first Nest Hub smart display from 2018. However, it's also introducing four new Google Home devices: three Nest cameras and a Google Home smart speaker.
You’ll have 10 voices to choose from for Gemini, and Google’s goal is that Gemini will allow natural, intuitive voice controls and conversational responses with more relevant details. During the demo in New York, I saw Gemini respond to a series of questions about fixing a dishwasher. Anish Kattukaran, Google’s Chief Product Officer for Home and Nest, asked Gemini about drainage issues, and Gemini offered a few common ways to fix the problem. When those ideas didn’t work, Gemini responded to follow-up questions with additional ideas, without needing the context again, and built upon the earlier responses.
Gemini also responded to complex queries like “which day is better for a barbecue this weekend?” and “add the ingredients for vegetarian pad thai to my shopping list,” with the latter prompting follow-up questions about preferences and other dietary restrictions.
When controlling the smart home, Gemini will be able to respond to multiple requests given in sequence, without needing to pause between them. You can tell it to “turn off the lights, start the vacuum, and lock the doors,” and it should comply. Before, you either needed to give those commands one at a time or group them in pairs and pause between each.
It should also recognize and respond to a wider variety of commands, so you won’t need to memorize an exact phrase to control a specific device, and it’ll add contextual awareness. If you tell it to “turn on the lights,” it should know that you’re talking to the smart speaker in the living room and respond accordingly by turning on just those lights. Gemini will also recognize exceptions, so you can tell it to turn off all lights except for the living room lamp, and it’ll comply.
Google has long aimed to make its smart speakers and smart displays conversational and intuitive to control. Replacing Google Assistant with Gemini feels like an important step in that direction. I’m sure it will respond well during most occasions, as Google Assistant was already capable of that. I’ll most want to test it when I’m tired and rambling before bed to see if it can make sense of my mumblings.
Gemini Live: Premium Storytime
To aid in bedtime, particularly with a younger audience, Google’s smart speakers and smart displays will also have access to a new feature called Gemini Live. This will allow you to start conversing with the AI without needing to repeat the wake word or stick to a specific cadence. You can just say “Hey Google, let’s chat,” and the assistant will continue listening and responding to your commands and questions, building off of what was already said, until you end the chat by saying “stop.”
During the demo, I saw Gemini keep up as Kattukaran interrupted it, changed the subject, and consistently reframed the question. With this mode, Gemini can participate in “storytime.” You can ask it for ideas or give it some of your own, and it can craft a story appropriate for a given age, either on its own or going back and forth with user-given ideas and prompts.
You can use the same feature to search for and get info on the music you want to play, without knowing much in the way of specifics beforehand. Gemini Live can help come up with projects or provide detailed info on a topic, while responding to follow-up questions and even reframing its answers based on who is listening. I saw it change its tune after it was told to explain something such that “my 4-year-old could understand.”
While the Gemini controls and conversational responses to smart home queries will remain free, Gemini Live will be part of a new premium tier of services for Google’s smart home. Gemini Live will cost $10 a month and is included as part of the Google AI Pro plan. This $10 standard tier includes important upgrades for the company’s security cameras, which are also getting an AI upgrade.
AI in Cameras: Adding Situational Awareness
In addition to Gemini Live, the $10 standard tier of Google’s premium smart home service includes 30 days of event video history and intelligent alerts based on recognized faces and objects. You’ll need to upgrade to the $20 tier to take advantage of most of the new AI capabilities coming to Google’s security cameras. That tier includes 60 days of history, as well as AI-generated notifications, AI descriptions of video clips, and an AI-generated recap of your day.
As with the Gemini upgrade, these changes will apply to current devices, new devices, and Google’s older cams, dating back to the indoor Nest Cam from 2015 and the first Nest Video Doorbell from 2018. With AI, the cams will help you filter through your notifications, and will have semantic scene understanding, so you can make specific searches and find specific clips.
For example, you’ll be able to search through the camera feed for clips of your gardener, or simply ask the app if the gardener came today, and it will understand, respond, and show the relevant clip. You can be more specific and ask if something ate the plants, or search for evidence that the kids left their toys on the driveway.
AI will be interpreting what’s happening in each clip to allow these searches, and can also populate generative descriptions of each clip for the sake of clarity as you scroll through them. You can form these clips into a custom daily briefing focused on your pets or kids. This Home Brief is customizable as far as time period as well, so you can ask to recap just the night while you slept or a morning where you were preoccupied with meetings so you can catch up on what happened while you were distracted.
Unlike Google’s smart speakers and smart displays, you won’t see much benefit from the AI in the cams in the free tier, but Google is expanding the free clip history from three to six hours.
Google Home App: Streamlined Design
The Google Home app will feature a new look, complete with three tabs designed for simple and easy navigation. Google is also bringing all older Nest devices, including the oldest thermostats and doorbells, into the Google Home app at last.
The main Home tab provides a customizable overview of your devices, allowing you to swipe between groups, including favorites and dedicated dashboards for other sets. The Activity tab displays all events that have occurred in your home, which you can streamline with the premium Home Brief feature, detailed above. The Automations tab displays your established routines and provides an easy way to create new ones, allowing you to set up tasks like having your lights turn on when you unlock the door, for example.
Above all of these tabs, you’ll see a search bar that allows for text and voice input. You can use this bar to take advantage of the Ask Home feature. Ask Home will respond to the same queries as Gemini in smart speakers, allowing you to build automations with your voice or simply find the controls for that one smart plug you forgot the name of. The Google Home app will also track usage, so you can ask it how long the lights were on and the door was left unlocked.
Most of the features of the app will be free, and you’ll even see animated previews of camera events in the activity tab for free. Creating automations with the Ask Home search bar will require a subscription to the $10-per-month standard tier.
How to Get Started
I’m looking forward to testing out these new features in my own home to see if Gemini makes a tangible difference over Google Assistant. You can try it yourself if you’re curious, as the early access period is now open. To do so, make sure your Google Home app is updated, then click the profile icon in the upper-right corner, and then Home Settings. Scroll down in this menu and look for Early Access and select the option to join. It might take a few minutes to process the request, but Google will send you a notification once the upgrade is ready to try.
We’ll also be testing all of Google’s new cameras for full reviews in the near term, as well as the new smart speaker when it launches this spring. Google’s new smart home seems promising, and with Amazon refining its own Alexa+ AI, the competition to make a more intelligent smart home is heating up. Stay tuned as we test these updates, and check out all of our current smart home favorites in the meantime.