“In the new future, every single interaction with the digital world will be through an AI assistant of some kind. We will be talking to these AI assistants all the time. Our entire digital diet will be mediated by AI systems,” Meta’s Chief AI Scientist Yann LeCun said at a recent Meta event. This bold prediction underscores a transformative shift in how we engage with technology, hinting at a future where AI personal assistants become indispensable in our daily lives.
LeCun’s vision is echoed across the tech industry. Demis Hassabis, CEO of Google DeepMind, emphasized their commitment to developing a universal agent for everyday life. He pointed out that this vision is the driving force behind Gemini, an AI designed to be multimodal from inception, capable of handling a diverse range of tasks and interactions.
These perspectives illustrate a consensus among leading AI researchers and developers: we are on the cusp of an era where AI personal assistants will significantly enhance both our personal and professional lives. Comparable to Tony Stark’s JARVIS, these AI systems are envisioned to seamlessly integrate into our routines, offering assistance and enhancing productivity in ways that were once the realm of science fiction.
However, to gauge our progress towards this ambitious goal, it is essential to first delineate what we expect from an AI personal assistant. Understanding these expectations provides a benchmark for evaluating current advancements and identifying areas that require further innovation.
If this in-depth educational content is useful for you, subscribe to our AI mailing list to be alerted when we release new material.
What We Expect from AI Personal Assistants
While certain features of an AI personal assistant might carry more weight than others, the following aspects form the foundation of an effective and useful assistant:
Intelligence and Accuracy. An AI personal assistant must be capable of delivering precise and reliable information, drawing from high-quality, credible sources. The assistant’s ability to comprehend and accurately respond to complex queries is essential for its effectiveness.
Transparency and Reliability. One critical expectation is the AI’s ability to acknowledge its limitations. When it lacks the information or is uncertain about an answer, it must clearly communicate this to the user, instead of ‘hallucinating.’ Otherwise, it doesn’t make much sense to have an assistant whose responses you always need to verify.
Multimodal Functionality. A robust AI personal assistant should be multimodal, capable of processing and understanding text, code, images, videos, and audio. This versatility ensures it can handle a wide range of tasks and inputs, making it highly adaptable and useful in various contexts.
Voice Accessibility. An AI assistant should be easily accessible via voice commands. It should respond quickly and naturally, mirroring the pace and quality of human communication. This instant accessibility enhances convenience and efficiency.
Real-time Streaming. The assistant should be always-on, omnipresent, and available across multiple channels. Whether through smartphones, smart speakers, or other connected devices, the AI must provide real-time assistance whenever and wherever needed.
Self-learning Abilities. You want your assistant to know your specific routines and preferences, but it is impractical to define exhaustive rules for every potential interaction. Therefore, an AI personal assistant should possess self-learning capabilities, allowing it to adapt and improve through interactions with a specific user. This personalized learning helps the assistant become increasingly effective over time
Autonomous Actions. Beyond providing information, a valuable AI assistant should have the autonomy to take action when necessary. This could include various tasks like managing calendars, making reservations, or sending emails, thereby streamlining tasks and reducing the user’s workload.
Security and Privacy. In an era where data security is paramount, AI personal assistants must ensure robust security measures. Users need confidence that their interactions and data are protected, maintaining their privacy and safeguarding against potential breaches.
Progress and Current Innovations
So where are we now? We obviously don’t yet have AI personal assistants that meet all the above criteria. But there are some tools that introduced significant breakthroughs in this area. Not surprisingly, they come from leading AI tech companies.
OpenAI’s GPT-4o
This May, OpenAI introduced their new flagship model, GPT-4o (“o” for “omni”). It marks a significant step towards more natural human-computer interaction. The model accepts input in any combination of text, audio, image, and video, and it can generate outputs in text, audio, and image formats. This multimodal capability positions GPT-4o as a versatile assistant for a variety of tasks.
Crucially, GPT-4o can be easily accessed via voice commands, supporting natural conversations with an impressive response time averaging 320 milliseconds, comparable to human interaction speeds. This accessibility and speed make it a strong candidate for real-time assistance in everyday scenarios.
In terms of intelligence, GPT-4o matches or exceeds the performance of GPT-4 Turbo, which currently leads many benchmarks. However, like other large language models, it remains prone to mistakes and hallucinations, limiting its use in tasks where accuracy is paramount. Despite these limitations, GPT-4o includes self-learning features, allowing it to improve responses based on user feedback. This partial self-learning ability helps it adapt to user preferences over time, though it is not yet as advanced as the personalized assistance envisioned in a JARVIS-like system.
While GPT-4o offers enhanced interaction capabilities, it does not perform autonomous tasks. Moreover, privacy remains a significant concern, as with many AI-powered tools, underscoring the need for robust security measures to protect user data.
Finally, OpenAI has not yet released GPT-4o with all the multimodal capabilities showcased in their demo videos. Currently, the public can only access the model with text and image inputs, and text outputs. Real-world testing of the model may uncover additional weaknesses.
Google’s Astra
Announced just a day after OpenAI’s GPT-4o, Google DeepMind’s Astra represents another significant leap in AI personal assistant technology. Astra responds to audio and video inputs in real time, much like GPT-4o, promising seamless interaction and immediate assistance.
The demo showcased Astra’s impressive capabilities: it could explain the functionality of a piece of code simply by observing someone’s screen through a smartphone camera, recognize a neighborhood by viewing the scenery from a window, and even “remember” the location of an object shown earlier in the video stream. Notably, part of the demo featured a user employing smart glasses instead of a phone, highlighting the potential for more integrated and innovative user experiences.
However, this remains an announcement, and the public does not yet have access to Astra. Thus, its real-world capabilities are still to be tested. It is likely that Astra, like other AI models, will still be prone to hallucinations and does not yet perform autonomous tasks. Nevertheless, the Google DeepMind team behind Astra has expressed a vision of developing a universal agent useful in everyday life, which suggests future iterations may include autonomous task performance.
Other Promising Players
As the race to develop advanced AI personal assistants heats up, several other major tech companies are making strategic moves, hinting at their imminent entries into this competitive arena. Although their next-generation AI personal assistants are yet to be launched, recent developments indicate significant progress.
Microsoft
Earlier this year, Microsoft acqui-hired Inflection, the company focused on developing “Pi, your personal AI.” While technically not an acquisition, Microsoft hired key staff members, including Mustafa Suleyman and Karen Simonyan, and paid approximately $650 million, mostly in the form of a licensing deal that makes Inflection’s models available for sale on the software giant’s Azure cloud service. Considering Mustafa Suleyman’s strong belief in personal artificial intelligence, this might be an indication that Microsoft is likely to offer its own personal AI assistant in the near future.
Amazon
Amazon, a pioneer in the voice assistant market with Alexa, remains committed to its mission of making Alexa “the world’s best personal assistant.” Recently, Amazon executed a strategy similar to Microsoft’s by hiring the co-founders and key employees of Adept AI, a startup known for developing AI-powered agents. The technology developed by Adept AI was licensed to Amazon, with the team joining Amazon’s AGI division to build real-world digital agents. Whether Amazon’s new product will cater primarily to enterprise customers or also introduce a personal AI assistant remains to be seen. However, integrating this technology could finally transform Alexa into a more powerful, conversational LLM-powered assistant. Currently, the old Alexa is hindering progress as Amazon has not yet figured out how to integrate the existing Alexa capabilities with the more advanced, conversational features touted for the new Alexa last fall.
Apple
Another leader in voice assistants, Apple, is also busy improving Siri. The company is partnering with OpenAI to power some of its AI features with ChatGPT technology, while also building its own models. Apple’s published research indicates a focus on small and efficient models, aiming to have all AI features running on-device, fully offline. Apple is also working on making the new AI-powered Siri more conversational and versatile, allowing users to control their apps with voice commands. For example, users will be able to ask the voice assistant to find information inside a particular email or even surface a photo of a specific friend. Apple places a strong emphasis on security, with the system automatically deciding whether to use on-device processing or contact Apple’s private cloud computing server to fulfill requests.
These strategic moves by Microsoft, Amazon, and Apple reflect a broader trend towards more sophisticated, user-friendly AI personal assistants. As these companies continue to innovate and develop their technologies, we can anticipate significant advancements in the capabilities and functionalities of AI personal assistants in the near future.
The Road Ahead
The race to develop the next generation of AI personal assistants is intensifying, with major tech companies like OpenAI, Google, Microsoft, Amazon, and Apple making significant strides. Each of these players brings unique innovations and perspectives, pushing the boundaries of what AI can achieve in our daily lives. While we are not yet at the point where AI personal assistants meet all the ideal criteria, the advancements we see today are promising steps toward a future where these digital companions become an integral part of our personal and professional lives. As the technology continues to evolve, the vision of having a truly intelligent, multimodal, and autonomous AI assistant appears closer than ever.
Enjoy this article? Sign up for more AI updates.
We’ll let you know when we release more summary articles like this one.