Augmented Reality: Beyond Conventional Time and Space

There’s a wonderful spirit of invention in the Augmented Reality (AR) community right now — the incredible experimentation and work being done with Apple’s ARkit is tremendously exciting. It’s truly thrilling to see AR accelerate both creatively and technically. Keep the demos coming!

Working with AR for 12 years now, it’s been amazing to experience AR’s evolution (read all about it and where things are headed in Augmented Human). It’s also been fun digging back into the archives recently to revisit some of my early AR prototypes from 2005 and later (especially the pre-iPhone projects and thinking about ARkit applications today).

Zach Lieberman’s ARkit experiments and process have been particularly inspiring to watch. Lieberman’s AR camera app test (video below) brought to mind artist David Hockney’s stunning and innovative photocollages — referred to as ‘joiners’ — from the 1980’s. Hockney’s joiners were a strong influence in my early AR prototypes. I was dazzled by Hockney’s approach to representing time and space, amplifying the abstraction and dynamism of Cubism, and building on the work of artists like Pablo Picasso and Georges Braque.

d1b17c5046bba561920f874e4422cebf-movement-photography-photography-portraits

Image: David Hockney, The Skater, 1984, photographic collage.

Hockney said of his photocollages, “I realized that this sort of picture came closer to how we actually see, which is to say, not all-at-once but rather in discrete, separate glimpses which we then build up into our continuous experience of the world”. Hockney presented a new way of seeing via the camera, one that mirrored the way we see in reality: through multiple glimpses that we piece together. AR, too, has the potential to rethink and present a new way of seeing and interaction with our world.

0d2d9235005e9940f6d26a5d8da98cb5

Image: David Hockney, Merced River, Yosemite Valley, Sept. 1982, photographic collage.

My AR Joiners series (2008-2009) paid homage to Hockney’s work. The AR Joiners widened Hockney’s concepts to use 2D video clips in AR with individual paper AR markers overlapping to create one larger AR collaged scene. The short video clips that composed the AR Joiners were each recorded over a series of separate moments (as opposed to one long video take that was cut into multiple fragments running on the same timeline). This was a conscious design choice: the AR Joiners were about the body moving in time, akin to Hockney’s photocollage process, with distinct moments and views, accumulating in a total memory of the space and experience across time. (Read more about the AR Joiners in a paper I presented at ISMAR 2009, Augmented Reality (AR) Joiners, A Novel Expanded Cinematic Form  published by IEEE.)

Hockney’s joiners, the AR Joiners, and the experiments we’re seeing today with ARkit  create new visual conventions beyond traditional time and space, all working toward building a novel language of AR. Another contemporary example is floatO, a photography iOS app that uses ARkit by artist Dan Monaghan.

In Architectures of the Senses: Neo-baroque Entertainment Spectacles (2003), Angela Ndalianis writes,

“The baroque’s difference from classical systems lies in the refusal to respect the limits of the frame. Instead, it intends to invade spaces in every direction, to perforate it, to become as one with all its possibilities.”

Ndalianis’s description of the baroque aligns quite nicely with Hockney’s joiners, the AR Joiners, and even Lieberman’s and Monaghan’s ARkit explorations; each of these works demonstrates ways of moving beyond the limits of the single photographic frame, expanding time in multiple directions, and puncturing conventional space.

But perforating the boundaries of reality doesn’t stop here: to truly grow the possibilities in AR, we will need to move past strictly vision-based experiments and engage the entire human sensorium with auditory, haptic, gustatory, olfactory, and visual experiences (in Augmented Human, there is a chapter dedicated to each of the senses and the opportunities with AR).

This is truly just the beginning of the dynamic, shape-shifting, and wonder-inducing new reality that is to come. I can’t wait to see, hear, touch, smell, and taste what’s next.

Let’s continue the conversation on Twitter: I’m @ARstories.

Augmented Human: How Technology is Shaping the New Reality is available from:

Amazon (USA)
Amazon (Canada)
Amazon (UK)
Indigo (Canada)
Barnes and Noble (USA)
Book Depository (Worldwide)
iBooks
Google Play

The Future of AR is… Sophisticated and Beautiful

This week, Wareable invited me to contribute to their Augmented Reality (AR) week feature. Here’s my vision for the future of AR:

“My prediction takes the form of my hopes and wishes for AR, and at its core what AR as an experience and a technology needs to be and do to truly advance.

The future of AR is sophisticated and beautiful. It enhances and is in sync with the physical world; it does not replace or supplant it. It does not overload; it aids and delights with elegance. It creates goodness, uplifting and enriching our lives. It ignites and invites curiosity and creativity. This is what we must strive for. May these new realities be deeply fulfilling and greatly benefit humanity.”

Thank you Wareable for including me and to each of the contributors for their thoughtful predictions. Read the full article here.

Last week, The Toronto Star interviewed me about the Art Gallery of Ontario’s (AGO) AR exhibit “Reblink.” I shared my thoughts on the importance of artists working with AR (which I go into more depth on in my book Augmented Human):

“Artists have the unique ability to take the ordinary and transform it into something extraordinary, and to show us the world in a completely new way. Augmented Reality does too. So AR and artists are a perfect match,” said Helen Papagiannis, an AR expert and author of Augmented Human: How Technology is Shaping the New Reality. “What’s next is an exploration of AR storytelling beyond just the visual: audio, touch, smell and taste.”

I can’t wait for you to read Augmented Human, in print September 2017. Here’s a post on why I wrote the book and who it’s for, with excerpts from the Preface.

augmented_human_comp_100dpiRGB

Purchase Augmented Human from:
Amazon (USA)
Amazon (Canada)
Amazon (UK)
Indigo (Canada)
Barnes and Noble (USA)
Book Depository (Worldwide)

Here’s Why I Wrote Augmented Human

Excerpts from Augmented Human: How Technology Is Shaping the New Reality:

Why I Wrote This Book

Twelve years ago, I caught my first glimpse of the power of Augmented Reality (AR) as a new communication medium. It was pure magic: a virtual 3-D cube appeared in my physical surroundings and I was awestruck. The augmented cube demo wasn’t interactive at the time (it did nothing else other than appear), however, it ignited my imagination for how AR could grow and evolve. At that moment, I dedicated my creative work, research, and public speaking to the new experiences AR made possible.

I wrote this book because I began to witness a much needed shift from a focus on the technology alone to a push toward creating compelling content and meaningful experiences in AR. This book is about exploring those big ideas and the extraordinary new reality AR affords. Now is the time to dream, design, and build our wondrous future.

As AR advances, we must ask: How can we design AR experiences to enhance and make a user’s life easier and better? MIT Media Lab founder Nicholas Negroponte said, “Computing is not about computers anymore. It is about living.” AR is no longer just about the technology, it’s about living in the real world, and creating magical and meaningful experiences that are human-centred. This book is about how AR will enrich our daily lives and extend humanity in unprecedented ways.

Who Should Read This Book

It’s not too often an entire new medium emerges. You should read this book if you’re a maker, a doer, and an explorer who is excited by creating a path where there is no trail, and want to contribute to this rapidly growing industry. You should also read this book as an informed consumer for a peek at the new experiences that will change the way we live, work, and play.

You are a designer, a developer, an entrepreneur, a student, an educator, a business leader, an artist, and a technology enthusiast curious about and excited by the possibilities AR presents. You are committed to designing and supporting AR experiences for the deepest of human values to have a profound impact on bettering humanity.

augmented_human_comp_100dpiRGB

Purchase Augmented Human from:
Amazon (USA)
Amazon (Canada)
Amazon (UK)
Indigo (Canada)
Barnes and Noble (USA)
Book Depository (Worldwide)
iBooks
Google Play