Augmented Reality, “Her”, and the Story of You

tumblr_my58njS7rs1seyhpmo1_1280Her pixel art by QuickHoney

Her is a story about people-centric technology. Spike Jonze shows us a near future where it’s all about you. This is our new Augmented Reality (AR), and it’s not science fiction.

I’ve been working with AR as a PhD researcher and designer for the past decade. The second wave of AR will surpass the current gimmickry and extend our human capacities to better understand, engage with, and experience our world in new ways. It will be human-centered and help to make our lives better. Driven by the one thing that is central and unique to AR – context – our devices will be highly cognizant of our constantly changing environments continually deciphering, translating, analyzing, and navigating to anticipate our specific needs, predicting and delivering personalized solutions with highly relevant content and experiences. Our smart devices will act on our behalf. This next wave of AR is adaptive; it is live and always on, working quietly in the background, presenting itself when necessary with the user forever at the center. It works for you, and you alone. It knows you very well, your behaviours, your likes, dislikes, your family and friends, even your vital statistics. The next wave of AR combines elements like Artificial Intelligence (A.I.), machine learning, sensors, calm computing, and data all to tell the unique story of you.

Meet Samantha, the world’s first intelligent operating system. Samantha is not real yet, only imagined in Jonze’s film Her; however, she gives us a glimpse of our soon to be augmented life when our devices come to learn and grow with us. Dr. Genevieve Bell, Director of Interaction and Experience Research at Intel, describes a world of computing where we enter a much more reciprocal relationship with technology where it begins to look after us, anticipating our needs, and doing things on our behalf. Dr. Bell’s predictions are echoed by Carolina Milanesi, Gartner’s Research Vice President. Milanesi states that by 2017, your smartphone will be smarter than you. “If there is heavy traffic, it will wake you up early for a meeting with your boss, or simply send an apology if it is a meeting with your colleague. The smartphone will gather contextual information from its calendar, its sensors, the user’s location and personal data.” Gartner’s research claims this will work with initial services being performed “automatically” to assist generally with menial tasks that are significantly time consuming such as time-bound events, like calendaring, or responding to mundane email messages. A gradual confidence will be built in the outsourcing of menial tasks to the smartphone with an expectation that consumers will become more accustomed to smartphone apps and services taking control of other aspects of their lives.

relationshipwithdevices_intelImages from Intel’s video interview with Dr. Genevieve Bell: What Will Personal Computers Be Like in 2020?

Gartner calls this the era of cognizant computing and identifies the four stages as: Sync Me, See Me, Know Me, Be Me. ‘Sync Me’ and ‘See Me’ are currently occurring, with ‘Know Me’ and ‘Be Me’ just ahead, as we see Samantha perform. ‘Sync Me’ stores copies of your digital assets, which are kept in sync across all contexts and end points. ‘See Me’ knows where you are currently and where you have been in both the real world and on the Internet, as well as understanding your mood and context to best provide services. ‘Know Me’ understands what you need and want, proactively and presents it to you with ‘Be Me’ as the final step where the smart device acts on your behalf based on learning. Samantha learns Theodore very well and with access to all of his emails, files, and other personal information, her tasks range from managing his calendar to gathering some of the love letters he ghostwrites to send them to a publisher, acting on his behalf.

Milanesi states, “Phones will become our secret digital agent, but only if we are willing to provide the information they require.” Privacy issues will certainly come into play, and a user’s level of comfort in sharing information. Dr. Bell observes that we will go beyond “an interaction” with technology to entering a trusting “relationship” with our devices. She reflects that a great deal of work goes into “getting goodness” out of our computing technology today and that we “have to tell it a tremendous amount.” She continues that in 10 years from now, our devices will know us in a very different way by being intuitive about who we are.

her-movie-2013-screenshot-samantha-pocketStill from Spike Jonze’s film Her

The world is filled with AR markers, no longer clearly distinguishable as black and white glyphs or QR code triggers; the world itself and everything in it is now one giant trackable: people, faces, emotions, voices, eye-movement, gesture, heart-rate, and more. The second wave of AR presents a brave new digital frontier, where the objects in our world are shape-shifting, invoked, and on-demand. This era will see one of new interaction design and user experiences in AR, towards natural user interfaces with heightened immediacy; we will be in the presence of the ‘thing’, more deeper immersed, yet simultaneously with both feet rooted in our physical reality. Our devices will not only get smaller and faster, and closer to, and perhaps even implanted inside our bodies, they will be smarter in how they connect with and speak to each other and multiple sensors to present a multi-modal AR experience across all devices.

Samantha is just this. She is a universal operating system that seamlessly and intelligently connects everything in her user Theodore’s world to help him be more human.

In a telephone conversation with Intel’s Futurist Brian David Johnson, he described to me how for decades our relationship with technology has been based on an input output model which has been command and control: if commands aren’t communicated correctly, or dare we have an accent, it breaks. Today, we are entering into intelligent relationships with technology. The computer knows you and how you are doing on any particular day and can deliver a personalized experience to increase your productivity. Johnson says this can, “Help us to be more human” and comments on how Samantha nurses Theodore back to having more human relationships. Johnson states that technology is just a tool: we design our tools and imbue them with our sense of humanity and our values. We can have the ability to design our machines to take care of the people we love, allowing us to extend our humanity. He calls this designing “our better angels”. Johnson says the question we need to ask is, “What are we optimizing for?” The answer needs to be to make people’s lives better, and I wholeheartedly agree.

My personal hopes for the new AR are that by entering into this more intelligent relationship with technology, we are freed to get back to human relationships and to doing what we love in the real world with real people, without our heads buried in screens. There is a whole beautiful tactile reality out there that AR can help us to explore and ‘see’ better, engaging with each other in more human ways. Get ready for a smarter, more human, and augmented you.

Let’s continue the conversation on Twitter: I’m @ARstories.

READ: “Augmented Human: How Technology is Shaping the New Reality.”


  1. The obvious concern involved with an augmented reality that permeates nearly every aspect of our daily lives is that of privacy. If we’ve learned anything since the initial NSA leaks, it’s that our devices are a vehicle through which personal information is made readily available to those with unhindered access (governments, hackers, etc.) In your studies, do you see any developments in efforts to secure personal information? Are we looking at an inevitable future of pure citizen transparency?


  2. Every time I hear about a new trick someone’s taught their smart phone, I’m reminded of a radio commercial that aired sometime back in the early 2000s. The voice claimed that “someday my phone will be able to do [such] and [such] and make me a sandwich!” The other two things have probably been accomplished by now, but I’m not upgrading until they come out with that sandwich phone!


  3. We can love a car, a favorite spot to sit and the feeling of a good piece of music. Why not the soothing voice of simulation which tells us what we want to hear.


  4. A nice read. Halfway through ‘her’ and looking forward to finishing it over the weekend.

    P.s. I turned off Siri after the five 5 mins of playing with it. Will certainly need more AR to be useful by 2017.


  5. I feel frustrated about this future coming towards us, Since starting my e marketing classes I can see this huge smile my teacher puts on every time we speak about this new type of technology, and every-time she asks us if we do not think if all of this is not wonderful I just feel resigned against people who seem have give up of their right to have a choice or opinion leaving it so someone else. Well but what about me I do not want someone else to think for me! As a personal choice I like to analyze, compare, discuss and take decisions, whatever they right or wrong. It seems I constantly hear the same excuse when trying to justify such an invasion in our lives, “It will leave us more time for the important things”. It is a brilliant idea but how is it going to work. How is this nation going to generate money by having more free time, and how countries that do not have access to the same technology are going to cope with a even bigger gap between evolution? Can you imagine to be born in such a future where a computer will analyze all the data they have on you and start to plan all your life. Witch school is best adapted to you, which sport you would be best, which friends are more compatible with you, which job you should chose. We just look at the bright side of this technology, yes it would make it easier to be wake up earlier to not arrive late, but the problem is that once we start to use this new technology we will become more and more dependent of it and let it take decisions that only us should take. And how are we going to react to distress when something goes wrong and we need think by ourselves. I already see people lost when suddenly their smartphone was left at home by mistake at home. I hate this marketing of a future when we do not need to worry about anything because someone will have already thought about it for you. And how will we become more friendly or social as we will not need to go out if our “best friend” will be at home, with such perfection at home I only see us becoming more demanding towards others.

    Descartes said: ” I think therefore I am”. If the human race does not want to think by themselves and have control of their own how will we find our own identity and how will we live our own life?


    1. I don’t feel like people have given up anything, because there’s still the choice whether or not to give said choice up. Technology is freedom, and one day, I hope humanity and robot-kind will be able to live as equals. Bob Marley style.


  6. The augmentation that is supposed to help, stresses me out more and more. Sure things are faster and more convenient but with that comes problems unseen, now here and irreversible. Sorta. Not sure I’m excited. Interesting read.


  7. Reblogged this on keyteachers and commented:
    ‘Her’ a film directed by Spike Jonze. Does technology extend our humanity and is it a tool that can help us to develop our human connections or is there the possibility of technology replacing them? This film is MA and I haven’t seen it but the trailer certainly provoked some deep thinking questions.


  8. Great read. Man will adapt to digital AR. A pair of self tinting no line bifocal glasses are an analog technology that we enter into a symbiotic relationship with. How many battery powered devices do you carry? 🙂


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.