I tried "window.ai" in Chrome

I never got chrome's built-in AI model working on Chrome's Canary release (maybe because I am on v128), so I tried Chrome's Dev Channel - thanks @theo.

Google Chrome has started shipping experimental AI features such as built-in AI in Chrome's Dev/Canary channels. In this tutorial, you will get to know how you can try the built-in AI model in chrome dev.

Installing Chrome Dev

Head over to chrome's dev release download page and install the dev release of chrome.

Setting up experimental flags

Once you have installed chrome dev, visit chrome://flags url and search for two flags:

• #optimization-guide-on-device-model - set it to Enabled BypassPerfRequirement
• #prompt-api-for-gemini-nano - set it to Enabled

Now, visit chrome://components and search for Optimization Guide On Device Model component. If its not showing up in the list, disable/enable the above flags, relaunch/restart chrome and keep doing this until you get that component.

Once you get that component, click check for updates (the initial version will be 0.0.0.0). It will start the download automatically and after it gets completed, you will see the updated version of that component (for me it was 2024.6.5.2205).

With that being done, you are now ready to use window.ai in chrome.

Creating a session

The initial API of window.ai provides two methods createTextSession and createGenericSession to create a new chat session. I am not aware of the differences between the two as there's little to no documentation about it, but for now you can go experimenting with them.

There are methods to verify if a session can be created or not as well. They are named as canCreateTextSession and canCreateGenericSession. They return a state which is used to determine when a session can be created. So, for example, if you haven't downloaded the model locally yet (remember that Optimization Guide On Device Model component), it will return a state named after-download which means a session can be created after you download the model.

let aiSession = await window.ai.createTextSession();
await aiSession.prompt("hey, how are you?");

Also, you can destroy the session by using the destroy() method on the session created.

aiSession.destroy();

Conclusion

I found the output to be pretty slow, but considering the fact that it's still in its initial stages, things can change, and it can depend on your machine's resources and configuration as well. The great thing about it is that it works offline as well. And I think that's the power of built-in AI models, but that does mean you will have to trade-off resources with localization unless the computation power increases.

Here's a brief and not so great conversation with the built-in AI in chrome.