Augmented Reality Top 8 of 2018
*UPDATE: VRScout published this article here. Special thanks to Jonathan Nafarrete, Editor in Chief of VRScout.
As a new communications medium, Augmented Reality (AR) is changing the way we consume and share knowledge and how we express ourselves creatively. This year AR included a range of experiences from contemporary art to currency to close-captioning. As a researcher and designer working with AR for the past 13 years, here are my picks for the Top 8 of 2018 (in no order of ranking).
These AR projects, apps, products, and tools are helping to rewrite reality and the way we engage with our immediate surroundings and world at large. Each example highlights one of three things I believe AR does really well: Visualization, Annotation, and Storytelling.
VISUALIZATION:
AR is a powerful visualization tool. It allows you to bring an object or concept into reality that is otherwise imagined, inaccessible, or difficult to grasp and can even help to make that which is invisible now visible.
1. Big Ben Snapchat AR Lens
Snapchat is using visualization to temporarily reveal something that is currently unavailable in reality: Big Ben. The popular landmark in London is under repair until 2021 and is currently covered with scaffolding. AR is used to remove the scaffolding giving Snapchat users a peek at the restored icon.
If you look closely at the video, you’ll see the clouds are replicated and added behind the virtual tower to mask the physical scaffolding and heighten the illusion. Industry veterans like Matt Miesnieks, CEO of 6D.ai, noticed these technical details commenting on a post I shared: “The way the sky and cloud is backfilled in where there is physical scaffolding is really impressive,” said Miesnieks. “Partial diminished reality.”
AR helps bring Big Ben back to life temporarily for the holidays with both sight and sound. A nice feature in the Snapchat Lens is the accurate time being shown on the Great Clock’s face, with bell chimes every 15 minutes and on the hour. Big Ben has been silent during the repairs and the clock hands have been removed. As one of London’s most recognizable and photographed landmarks, the restoration and maintenance has left many tourists disappointed. Now with AR you won’t have to wait until 2021 to snap that selfie: the front-facing camera also adds a virtual hat on users. Tourists rejoice!
2. Notable Women, by former Treasurer of the United States Rosie Rios, Google Creative Lab, and Nexus Studios
The Notable Women AR app helps you discover the accomplishments of activists, artists, scientists, business leaders, writers, civic leaders and more—right on the money in your wallet. Visualization is used to see 100 historic American women where they’ve historically been left out: on U.S. currency. AR is used to “swap out the faces we know for the faces we all should,” as the Notable Women website reads.
One of these women is Sojourner Truth who campaigned for abolition and women’s rights, becoming famous for her “Ain’t I a Woman” speech at a women’s rights convention in 1851. Named Isabella when she was born into slavery, she changed her name after gaining her freedom.
The 100 historic women were selected from the Teachers Righting History database, a collection of women whom the American people recommended to appear on actual U.S. currency during Rosie Rios’s time at the U.S. Department of Treasury. While the app is designed with teachers and their students in mind (lesson plans are available here), the content is for everyone.
3. Step Inside the Thai Cave in Augmented Reality, The New York Times
This example from The New York Times proves AR doesn’t have to be overly complicated to be effective — the use of AR was extremely powerful in its simplicity. The AR story visualized to scale the size of the claustrophobic cave openings rescuers traversed. Part of the brilliance of the visualization experience was that you could see these tunnels in reference to the size of your own body: this immediately made it personal.
“The NYT AR piece on the cave rescue was an excellent application of the medium,” said Jon Wiley, Director Google AR VR. “My son, who is about the same age as the kids that were trapped, had been hearing about this story and we talked about the challenges. The AR experience, where we could see the exact scale of the cave passage was fascinating, allowing us to see for ourselves just what a terrifying challenge it was to rescue those kids.”
Graham Roberts is the director of immersive platforms storytelling at NYT. “We add value with the AR moments by keeping them extremely simple and allowing them to do one thing well,” Roberts said. “In this case, being able to present cave slices from a survey in real scale projected into the context of your immediate environment. This is something that could not be done without this technology, and gives a more intuitive understanding of an inherently spatial story.”
ANNOTATION:
Annotation with AR helps guide you through the completion of a task, navigate a new environment, or even provide real-time descriptions of what’s happening around you.
4. Smart Caption Glasses, National Theatre, Accenture, Epson
London’s National Theatre is using AR to help make its performances more accessible for people who are deaf and hard of hearing. When wearing a pair of glasses (designed and manufactured by Epson) users see a transcript of the dialogue and descriptions of the sound from a performance displayed on the lenses in real time.
Audience members with hearing loss previously relied on dialogue screens at the side of the auditorium (only available for a handful of performances) where you had to switch your attention from the stage to the captions on the screen. A member of the audience named Deepa who tried the smart caption glasses at the National Theatre commented, “I thought that the freedom to be able to read the captions and see the actors’ face was a big improvement. I’ve never felt that I was able to do that until I wore the glasses.”
The glasses are currently available for the productions “War Horse” and “Hadestown” and from January 2019 they will be available for booking on most new shows in the venue’s three theatres. Details on how to book the glasses are available here.
5. Bose AR
Bose also wants to annotate the world around you using a pair of AR glasses (called Frames), but they want to do that with audio instead of visuals. “Bose AR represents a new kind of augmented reality — one that’s made for anyone and every day,” said John Gordon, vice president of the Consumer Electronics Division at Bose. “It places audio in your surroundings, not digital images, so you can focus on the amazing world around you — rather than a tiny display. It knows which way you’re facing, and can instantly connect that place and time with endless possibilities for travel, learning, music and more.”
This week the company shared more details on Frames and the Bose AR SDKwith developers in a webinar led by Chuck Freedman, Developer Relations Lead at Bose. Three key areas were highlighted: travel, fitness, and gaming.
Freedman noted how Frames enable you to be heads up and hands free when travelling, absorbing the sights and sounds of the city you are in with the glasses telling you what you’re looking at pointing out the significance of buildings and perhaps even having a local to virtually accompany you (in April 2018 Bose acquired Andrew Mason’s walking tour startup Detour, a company featured in my book “Augmented Human“). Bose is also working with companies on fitness coaching experiences, unlocking data not available before like detecting rep counts a person does and how long a pose is held for helping you to meet your workout goals. Freedman hinted at a coming announcement of aligning with a prominent indie game developer and also referenced a pitch competition that will launch in January.
STORYTELLING
AR makes possible new modes of storytelling and creative expression with experiences unfolding in both our homes and public spaces — it changes the way we tell, share, and even remember stories.
6. ReBlink at the Art Gallery of Ontario (AGO) by Impossible Things
Tim Cook tours ReBlink at the Art Gallery of Ontario. Photo: Apple
ReBlink blends traditional art with AR to remix classic paintings in the Art Gallery of Ontario‘s permanent collection. Using a custom app for smartphones and tablets, visitors use their device’s camera to unlock artist Alex Mayhew’s modern twists on historical works of art. One of the things I particularly loved about ReBlink was the artful conversation between the old and the new and the delightful details hidden in the reimagining of the works of art. I was surprised and impressed by the three-dimensional quality of the AR work; despite the original paintings being flat, the AR content had an incredible amount of depth as though you were peering into a diorama. The paintings came alive with AR.
Artists have the unique ability to take the ordinary and transform it into something extraordinary, and to show us the world in a completely new way. AR does too, so AR and artists are a perfect match and that’s exemplified in ReBlink.
7. Enter The Room, The International Committee of the Red Cross (ICRC) in partnership with Nedd
Enter The Room sparks a dialogue about war; it’s an app that hopes to build empathy. Enter The Room was featured by Apple’s editorial team on the US App Store highlighting how the app “illustrates the devastating effects of war without depicting soldiers, weapons, or even the victims of violence.” The story is instead told through the room of a child and unfolds over the course of four years.
Using your iOS device, you walk through a virtual doorway that appears in your space. You find yourself in a child’s bedroom where war wages just outside the window. With AR, this room was in my home now too. I was there. And because I didn’t have the ability to control anything in the room like I might in a game, I felt helpless and afraid. I empathized with the effects of war on that child and family.
In February I wrote an article on “Augmented Reality Storytelling: The Body and Memory Making.“ I described how the emotions from an AR story stay with you long after you put your device down, or take the headset off, transforming into memories that are virtually scribed onto your environment, like an augmented palimpsest. We’re no longer just designing stories, we’re now designing memories with AR. AR stories leave a virtual imprint in our physical spaces and this is part of Enter The Room’s power.
8. Project Aero, Adobe
Project Aero is an AR authoring tool from Adobe that makes it easier for designers to create immersive content using popular tools they are already deeply familiar with such as Adobe Photoshop CC and Dimension CC. Project Aero is currently in private beta (you can request early access here).
I’m thrilled to be among the first artists and designers to create AR experiences with Project Aero (you can see some of my work in my Instagram posts and stories). I love how fast and easy it is to bring ideas from my imagination to life. Tools like Project Aero are a big step in helping to democratize AR as a new creative medium.
Real or AR and Augmented Human in 2018
Personal AR victories for 2018 included starting my game “Real or AR” on Instagram, which I play with audiences from around the world in my Instagram story and on global stages at my keynotes (hello to my new friends in Mexico, Finland, Japan, England, Brazil, Belgium, Portugal, Canada, and the USA!). Wagner James Au wrote an excellent piece about the game here. Follow @AugmentedHuman on Instagram and play along to help strengthen your virtual muscle for 2019 and beyond.
Also in 2018 my book Augmented Human (published by O’Reilly) was translated into Korean, Chinese traditional script, and was just released in India. Mike McCready, President of the Alberta Chapter of the VR/AR Association, has started a VR Book Club and “Augmented Human” will be the first book readers will meet to discuss in VR in January 2019. Join people from 35 cities and 5 continents by RSVP’ing here.
Congrats to the AR teams listed above in the Top 8 of 2018 and to the community and industry working hard to elevate AR as a medium that extends, enriches, and enhances the world around us.
Need help making your AR work the best of 2019? I consult individuals and organizations at all project stages as an AR subject matter expert. Get in touch.
PURCHASE “AUGMENTED HUMAN” BOOK: ENGLISH: AMAZON (US), AMAZON (CA), AMAZON (UK), ITUNES, GOOGLE PLAY
KOREAN TRANSLATION: ACORN PRESS, CHINESE TRADITIONAL SCRIPT TRANSLATION: GOTOP, ALSO AVAILABLE IN INDIA: SHROFF PUBLISHERS