The rise of AI means computers are better at understanding us and finding what we want than ever before. As a result, big tech companies like Apple and OpenAI are offering new ways of interacting with computers that bypass traditional displays, mice and keyboards. Will screens soon become a thing of the past?
Although large language models such as ChatGPT began as software, companies are now exploring how these tools might be made more physical. OpenAI, along with ex-Apple designer Jony Ive, is the “iPhone of AI”, a physical device to house ChatGPT.
Last year, the start-up company released its Ai Pin, a screenless personal assistant powered by ChatGPT that you can interact with using your voice. At the 2024 Consumer Electronics Show this week in Las Vegas, Nevada, another start-up announced a voice-controlled device called the R1 that uses AI to control your apps.
Advertisement
Meanwhile, Apple, though not strictly an AI company, is releasing its first virtual reality headset, the Vision Pro, which it promises will usher in an era of “spatial computing”. Users can see digital content blended with their physical space and navigate it using their eyes, hands and voice.
Whether Apple, OpenAI or a smaller start-up will break our decades-long screen fixation depends on who you ask, but the promise of screenless computing isn’t new, says at the University of Oxford. Humane’s Ai Pin, which promises to intelligently sort through and analyse anything you record while using it, is similar to personal assistant devices proposed , he says, and the possibility of a screenless future has been a consistent theme in the history of computing ever since.
However, using AI to power these personal computing devices offers capabilities that weren’t available before. “On one hand, that’s really exciting,” says Van Kleek. “On the other hand, I think that we’ve got a long way to go. These sorts of systems are proof of concept and slightly gimmicky.”
While the technology behind AI and language models has made enormous progress in recent years, there hasn’t been corresponding progress in how we might physically interact and have human-like conversations with these systems, says at University College London. “Eventually we will still hit this fundamental problem, which is that when I’m speaking, it’s often a mess, and we have to do a dance to figure that out,” he says. “It’s nowhere near as precise as using a mouse to click a button on a screen.”
Screens and text make use of vision, our most powerful sense, and they let us interrogate and check information in a way we can’t with audio, says at the University of Southampton, UK. This is especially true if the conversation is siloed within the device. “If you start a conversation with a device, you should be able to somehow continue it on another device,” he says.
For truly fluent conversation and interaction, these AI devices will also need to remember context as well as have mutual assumptions and beliefs shared with the user, a process that psychologists call grounding, says Brumby. While ChatGPT is better at this than most voice-powered assistants, it is still some way off the level that would be needed to use the technology without running into problems, similar to when Siri can’t play the song you’re asking it to.
Despite the staying power of screens and written text, many people have already comfortably moved into a screenless world for some tasks. For example, millions of people interact with Siri while driving, especially in the US, where traffic is bad, says Van Kleek. This could mean a future in which people adopt both screens and screenless interactions. “It’s going to go in both directions simultaneously, depending on what you need at the time,” says Hegland.
