Get in touch from Matthijs Munnik on Vimeo.
In between performance and installation, this living interface
A short history of augmented reality
Matthijs Munnik
In this essay I will give a short introduction into the history of Augmented Reality, discussing the higlights of a history that started 30,000 years before Christ. After this short introduction I’ll write about my own project and how it relates to this rich history of science and art.
This story begins with the start of the history of painting, cave paintings in the Chauvet Cave in South France. These paintings are the oldest known to man and consist of a wide variety of animals, ranging from horses, lions and owls to rhinos. These drawings are extremely vivid and and even today one can easily identify each specie.
Dangerous animals – mammoths, lions cavebears and whoolly rhinoceros – have great prominence in the Chauvet cave and were also depicted much more often in other places, than more innocent and common creatures as hares, birds and fish.
We can only speculate about the reason for the paintings, but one can imagine that viewing these images must have been a spectacular and mystical experience. Many researchers believe that these paintings were used for shamanic rituals and regard the caves as holy sanctuaries. The cave would be like a shadow realm, thus being the first man made virtual reality.
From here we make a huge leap, from a prehistoric sanctuary to the famous egyptian pyramids. From the earliest periods of Egyptian history, all Egyptians were buried with at least some burial goods which they thought necessary after death; food, pottery, combs and sometimes furniture and even servants for richer citizens. After a while these real products were replaced by woorden models, depicting activities the deceased could do in the afterlife. This was possible because egyptians saw these paintings as a form of magic, to them an image of a servant was the same as a real servant. This was of course also very practical, as one wouldn’t have to waste servants and products, because you could use images instead.
While for regular citizens a small burial might do the trick, for the pharaoh this was a different story. A giant pyramid would be erected to hold the sacraments of the pharaoh and he would be buried deep inside, in a grave chamber together with all his burial goods. After the burial the pharaoh’s spirit would leave the pyramid and cross to the eternal afterlife with all these possesions.
For the Egyptians these grave chambers were a passageway to the afterlife, so they can be seen as shadow realms as well. A world that is both real and unreal, created by the magic of drawings. In that sense you could say these gravechambers, just like the cave paintings are a form of alternate reality. Of course there are many more likewise examples in the history of art around the idea of taking burial goods into the afterlife.
Egyptian paintings were constructed following strict rules, resulting in a more schematic rather than realistic representation. However this wasn’t a problem in Egyptian society, on the contrary it was needed for the “magic” of the painting to work properly. Only a rightly constructed image, following the correct template would be a “real” depiction. The most important figures are often shown as the highest in a composition, also from hieratic motives, leading to the "vertical perspective", common in the art of Ancient Egypt, where a group of "nearer" figures are shown below the larger figure or figures.
It would not be until the 5th century BC that systematic attempts to evolve a system of perspective begun, in ancient Greece, as part of a developing interest in illusionism called skenographia, wich translates as scene painting. These scenes were used in ancient Greek theatres as a backgrounds for the plays. Images painted on a flat panel, usually representing the façade of a palace with columns and side porches with doors, drawn in a form of perspective. These paintings were mounted on wooden frames, which were placed in front of the stage-building.
The philosophers Anaxagoras and Democritus worked out geometric theories of perspective for use with skenographia. Mural paintings based on skenographia have been found in houses too, for example in Pompeii, proving the art was not only confined to a stage. What differs skenographia from our previous examples is that this time the image is meant to create an actual illusion of reality for entertainment’s sake, instead of a schematic depiction for religious reasons. This switch from sacred images to images for entertainment was really important for the development of augmented reality later on, as we will see in the upcoming examples.
Unfortunately in the Late Antique period use of perspective techniques declined. Early medieval art was slow in relearning previously known models for perspective, as art was used for spiritual reasons again instead of entertainment, illusions of reality became less important than schematics.
The first modern optical basis of perspective was layed down by an Iraqi physicist going by the name of Alhazen, who was the first to discover that light projects conically into the eye. He wrote his findings down in his Book of Optics. His conclusion would have theoretically been enough to construct a model for perspective, but Alhazen was not concerned with painting. It wasn’t until 1413, that Filippo Brunelleschi demonstrated the geometrical method of perspective, used today by artists, by painting the outlines of various Florentine buildings onto a mirror. After that discovery, this method took Italy by storm.
Before we dive further into optical illusions I’ll go back in time a bit to show another, yet different approach to augmented reality. Thus far we’ve only discussed static images, now we’ll have a look at the first animation, in the form of a shadow puppetry. Shadow puppetry originated during the Han Dynasty, between 206 BC–220 AD, when one of the concubines of Emperor Wu of Han died. The emperor was struck with grief and asked his court officers to bring her back to life. Because you can’t deny the emperor of China anything, they made a shape of the concubine in leather, constructing her joints in a way they could be animated and adorned with painted clothes. Using an oil lamp they made her shadow move, bringing her back to life.
With this shadow they actively created another world, not only for ritual reasons but also for entertainment. As shadow plays became more popular, during the ming dinasty more than 40 groups were active in Beijing alone, they spread out troughout Asia and Eurasia by conquering mongols, and eventualy spread to Europe in the mid-18th century, when French missionaries in China took it back to France in 1767 and put on performances in Paris and Marseilles, causing quite a stir. These shadowplays were quickly combined with the technique of magic lanterns, to create a Phantasmagoria, a ghost projection show.
These magic lanterns are another important aspect in the history of augmented reality. It’s the ancestor of the mordern slide projector, first described by Giovanni Battista della Porta in 1558. With an oil lamp and a lens, images painted on glass plates could be projected on to a suitable screen. During the 17th and 18th centuries the Magic Lantern was improved with the use of optics, enabling to create better lenses and thus better projections.
The slides often consisted of scary images, demons, skeletons and hellfire. Most likely due to the christian tradition of creating fear of eternal damnation. However also more friendly pictures, such as landscapes and harbours were projected, later animated with cleverly constructed slides.
Magic Lantern shows were performed by travelling artists, showing their special shows troughout Europe, creating exciting shows around these projections. These shows were called Phantasmagoria, as I earlier mentioned. Some artist would pretend to summon the souls of the passed away, to great horror of the superstitious audience, actively augmenting reality. Instead of passive wall paintings or schematic shadows, these projections were quite realistic, creating an even more believable new reality.
After awhile these magic lantern shows became out of fashion with the introduction of a new technology; film. Even though one might argue that film also counts as an alternated reaity, my opinion is that it’s not. What differs the Phantasmagoria from projected movies is that the entertainers of the magic lantern created the illusion that there was really a ghost. To me this is different than a movie screen, which stays a passive screen, just like a television today.
Even though there are a few exceptions, for example the story of the audience running out of the cinema when they were showed the first movie of a train, because they thought it was going to hit them. I won’t discuss the rest of film history in this essay however.
Instead, I’ll go back to something I do regard as augmented reality, namely illusions. We’re back at the topic of perspective, around the 15th century this time. Since Alhazen’s book of optics perspective had made quite a development. After its perfection artists started to think of more complicated methods of projection. One of the most significant examples of the use of perspective to create a more elaborate illusion of space is anamorphosis.
Leonardo’s Eye, by Leonardo da Vinci, is the earliest known example of anamorphosis. Anamorphosis only works from one viewpoint, where it creates the illusion of 3d in the painted object. From any other viewpoint however the image looks distorted.
A famous and good example of the use of anamorphosis to augment reality is the story of the dome of the Church of St. Ignazio in Rome, painted by Andrea Pozzo. Because the church couldn’t construct a physical dome atop of the church, they asked Pozzo to paint one as an illusion on the flat ceiling. Pozzo calculated this impressive piece of perspective distortion to create an extremely realistic effect of both a dome and a ceiling leading up to heaven. Even though it only works from one spot, it still actively augments reality, this time combining spiritualy with realism and modern technologies.
Another way perspective was used to create a virtual reality, were the so called perspective boxes. These boxes were quite rare and still are, there are only six left in the world. They consisted of small wooden boxes with a peephole. Painted on the inside was, in perspective, a realistic view of a space or landscape. Looking trough the peephole gave the illusion of actually being in this space. They were made around the the 17th century, one of the most famous ones by the dutch artist Pieter Janssens Elinga. These boxes did not only augment reality, but actually tried to replace it with a virtual one, taking the illusion a step further.
It didn’t get this close to actual virtual reality until the discovery of the computer. The first computer, the Z1 constructed in 1938 by Konrad Zuse was only used for calculations. The earliest known examples of immersive, multi-sensory technology was not a computer, but a mechanical device, called the Sensorama. This machine, created in 1962 by Morton Heilig, as a prototype for his theory of future cinema. He wanted to encompass all of the spectators senses and created 5 prototype movies for his machine. Most noted is a movie of a bike ride trough Brooklyn, shown in stereoscopic 3D in wide angle view, provided with body tilting, stereo sound and tracks for wind and aromas to be triggered during the film. Even though this can be considered as a milestone in the history of virtual reality, Heilig was unable to get funding for his projects and the Sensorama work was halted and today remains primarily a curiosity.
After sensorama the first other breaktrough happened in the year ’68, when Ivan Edward Sutherland created the first virtual reality and augmented reality head-mounted display system. However this system was extremely primitive and heavy, so heavy in fact, that it had to be suspended from the ceiling.
Five years later there would be another keypoint in the now accelerating development of augemented reality, when Myron Krueger created video place. This was an interactive environment, wich didn’t require the user to wear any gloves or headmounted displays, but reacted on their movements and actions on a projected screen. Krueger was also the first to coin the term artificial reality, to describe his piece.
It was only in ‘ 89 the term virtual reality was coined, by Tom Caudell. Ten years later Hirokazu Kato developed the ARToolKit at the HITLab in Seattle. This toolkit is a vision tracking library that allows for the creation of augmented reality applications. Because this toolkit was made opensource it gave many programmers the opportunity to experiment with augmented reality.
Since then development of augmented reality systems has really taken off, with new projects emerging everyday, thanks to opensource libraries and easy accesible technology such as webcams, iphones or the latest google android phone.
