To get an understanding of what augmented reality is, how it's used and where it's going I first had a lot of research to do.
HISTORY
To begin I looked at the history of augmented reality
and
This was really useful as it explains AR from the very beginning. In 1957 by Morton Helig who created a machine that plays with the human senses to give the illusion of being somewhere else. This can be considered more 'virtual reality' and same in 1966 when Professor Ivan Sutherland of Electrical Engineering at Harvard University created a head mounted display, this was unfortunately far to heavy for the head and was suspended from the roof.
The first actual functioning AR system was for the US Air Force known as "Virtual Fixtures" where fixtures were what he described as cues to help guide the user in their task. It was only here when Augmented Reality was really born, where users would not only experience virtual aid, but it'll help them in real time. Virtual instruction coming up on screen or 'ghost' images displayed overlapping the real world image, this was what opened the door of creativity and allowed for endless possibilities.
The 90s was the experimental era for augmented reality, where
scientists would spend a lot of money creating equipment and software for
prototypes that were rarely even known about. This all changed in 1999 when
Hirokazu Kato of the Nara Institute of Science And Technology released the ‘ARToolKit’ to the community. This
allowed video capture tracking of the real world to interact with virtual
objects and allowed 3D graphics to overlay any OS platform.
It was this piece of
software which is responsible for the spark of augmented reality for years to come;
anything could be created with a simple marker card for tracking, a video
capture device and something to view it with. This again became even more
popular in 2008 when augmented reality was taken to smart-phones apps, devices
that could be purchased by the public and already had all of the hardware needed
means that creating and using augmented reality has never been easier.
WHAT IS IT?
I also needed to find out exactly what Augmented Reality was, what defined it and made it unique from virtual reality. I looked at
and
Here it explains that in it's simplest terms Augmented Reality is the super-imposing of a computer generated virtual image, onto a live feed of the world. This opens up a huge world of possibilities and questions which I know have to look into.
I also looked at
which was very helpful as it contained a large amount of information on history, technology, hardware and applications throughout the years.
HOW'S IT WORK?
So now I know a little about the history of Augmented reality and what it is, now I decided to look into how it works. I checked out
where it pretty much states that Augmented Reality works by having a device that can capture live footage and stream it back with a visual output, such as a webcam and monitor or phone camera and screen. Some sort of software would be needed to allow the object or world to be recognised by the device, then once an object has been recognised (For example the baseball card in the above link) a virtual image can be displayed over it (In this case the baseball player on the card).
Another way of tracking the object the user wants to augment, doesn't have to be purely visual with colours or cards. Augmented Reality can also be developed to respond to sensors on devices such as gyroscopes that help calculate orientation and rotation.
TECHNOLOGY
Now the fun part of my research, now I had to go find out the technology behind Augmented Reality, and most importantly the potential. This is where I look at loads of different videos to see what aspects of Augmented Reality exist and what is in development.So to start of:
This video shows off the basic concepts of AR, a camera picking up a card that I'll use to track and show a 3D model. The cards are usually back and white to show clearly and are easily recognised and allow of the tracking of depth and perception.
After looking into more videos I found this:
This I found interesting because it recognises a map full of colour and diversity and yet still projects the image and tracking very well. This could be great if when driving a passenger is able to give direction more accurately with a 3D map.
Also driving related, if you drive alone and don't have a passenger then maybe a Sat-Nav:
The good thing about this App is it in your phone, no need for a separate expensive Sat-Nav that shows coloured lines for roads and plain green and browns for land, this piece of AR allows the driver to clearly see whats in front of him exactly the same on their screen, with the addition of directions.
The reason behind most AR inventions seem to be to aid man in tasks he may have not been previously able to do without help, AR is that help.
This video is a great example of a man being helped to dissemble a BMW's parts with the help of a visual guidance system. This is of course is still in development and concept states but it still is a brilliant step forward. The fact something may be able to help find a mechanical fault and guide you to fix it without the need of an expensive mechanic or large car manuals and experience.
Another interesting thing I found was this really cool application
It allows the user to try on items of clothing that interact with movement and physics of the user, this is a great example of try before you buy and also adds a new way to see clothing from all angles, and more importantly, on yourself. The downside is a 3D model would have to created of all the available items of clothing, potentially in different sizes and also weighted for the animation. I think some people would prefer to see real images as opposed to the virtual ones, having said that though there is no reason a site couldn't feature both.
It's always best if you can try things out before you buy, especially with expensive things, for example houses...
This is a really quick and easy way for those who want to design their own house. Rather than spending lots of time trying to create a model using programs like 3DS MAX or getting someone else to. Here they can simply draw their design and see it come to life with lots of room of adjustments.
What about multiple floors?
Well there can be multiple cards that can join together to show the overall project, time consuming tasks can be concepted and quickly completed.
Further research also shows that it can all be done using just one card
Imagine this sort of technology for future 'The Sims' products, being able to pull out your house out of a pocket and play the game on a train or where you'd like. Maybe placing next to your friends house and having them interact with one another. It's concepts like these that bring a new social element into gaming, without the cliché of anti-social geeks locked away in there rooms.
Even if you're not aiming to build a house, AR can also be used for marketing hotel rooms.
This would be a great idea to stop those unfortunate disappointments when travelling hundreds of miles to a room that isn't how you expected.
Augmented Reality has many different benefits from looking at objects and trying out new things, but somethings are still missing and are a long way from completion, which got me thinking about games. The previous videos are all interesting in there own right, but wouldn't really make a very interesting game as they are, and I wondered how games would be designed with such technology.
This game works off the basics needed for AR
The game itself unfortunately never left Dev stage due to Gizmondo failing, so this game never reached it's potential. Games like this would be really fun to play with a friend, being able to create downs and cities, then defend or destroy them like in Black and White 2.
I found this game used a quite interesting use of markers
levelHead v1.0, 3 cube speed-run (spoiler!) from Julian Oliver on Vimeo.
I found 'LevelHead' a really interesting concept as it used just one cube to feature multiple levels, also when multiple cubes are introduced the character can be passed to another cube. Considering that it is essentially just six markers stuck on a cube, its cool how the software links them all together.
Another example is ARhrrrr
ARhrrrr is basically another zombie shooter but this ones special thanks to Augmented Reality. The player can rotate around buildings by simply moving the camera around, giving a perspective similar to that of a sniper in a helicopter. The player must then shoot zombies protecting civilians, once again very simple concept however I'd imagine surprisingly fun.
This seems like a really cool idea, a turn based AR game
Augmented Reality Toys.v2 (Work in progress) from Frantz Lasorne on Vimeo.
While an old tech demo that didn't seemed to ever get completed, the concept still has potential for those who would like to use their real life models in a game.
The game The Eye of Judgement is a good example of a working turned based AR game that allow use of physical cards into the game world.
Another use of a card character brought to life are the TOPPS 2009 baseball cards that when shown to the camera bring that baseball player on screen.
After the baseball player is shown on screen a mini game can then be played with them depending on the role of the player, such as batting or pitching.
This game would appeal very much to those interested in trading card games. The game play itself has some depth and isn't simplified so suit a more casual market and with the addition of being able to see your actual creatures come to life and duel gives the game much more action and stimulation than a standard board game.
Not all of the games I looked into would appeal those who enjoy shooting zombies or building houses, instead AR can be used to also entertain children
Children that always want to see the pictures when being read a story can now see in a completely different way, sparking their creativity like never before. This could also be taken one step further and used not just for bed-time stories, but also factual educational books. Think dinosaur skeletons coming to life when reading about them.
When researching into more interactive books I found this
Story books could become very interactive with technology like this, not only could you enjoy moving illustrations but actually interact to enhance and continue the story.
I also looked into more advanced AR games that are currently available and also those still in development. To begin I looked at EyePet on the ps3.
The eye pet was great for getting the younger audience more used to AR. It allowed the audience that love looking after virtual pets like tamagotchi and nintendogs to be able to see themselves on screen physically interacting with the Eyepet.
It seems that the quality of AR depends largely on the hardware used rather than the software. After some researching I found this.
This is a great example of AR tracking and responding at a very good rate, showing that it's possible to have very smooth feedback.
One limitation I've noticed so far however is that for the tracking and movement, some kind of marker or card is needed for the computer generated image to be displayed on. Further research has found that this limitation is becoming a thing of the past as a marker is no longer being needed.
The PSVita boasts the ability to use AR without the need of a marker, and doing so with great graphics and accuracy too. When using a marker however it enables a much larger scope of play with the WAAR (Wide-area-augmented-reality) which allows for use of upto 6 cards at once.
Reality Fighters shows how AR can work without the need of a marker card
Reality fighters doesn't just paste the fighters over the screen, the console can be moved and rotated which will change the angle on screen. Being able to play a fighting game on any stage with custom characters looks like huge fun!
I found this video which shows an AR program recognising and tracking surfaces in real time.
The impressive thing isn't just that it's able to track the surfaces in basic form, but that it's doing it in the some of the worst conditions for AR such as poor flickering lighting, no markers or textures and lots of moving parts including his hand. This video was created a few years ago so no doubt the tech has improved a lot since then.
I found this video when looking more in AR without the use of markers
The video shows a virtual car being remotely driven in the real world, the software is able to recognise depth and display the car behind objects as it drives past. The car is also not bound to one area on-screen and can drive around a garden marker less even when the camera is panning across.
Another great video I found shows a program recognising a real world object
Here you can see a castle that the software recognises and created a 3D version, which physics are then attached to so that reality can interact with virtual reality. The grey castle is just to show that the computer can track the castle constantly. The part I found really impressive was when he lifted the castle and the marbles fell, they still landed on the table to prove that the physics are restricted to just the castle's floor.
An advancement on the previous video shows another example how these physics can work
This video clearly shows a virtual car physically interacting with the real world, even when the car is stationary the object will move the vehicle. What's more impressive is when the table is banged the car shakes and bounces. I'm thinking of ''Godzilla Attacks Tokyo Augmented Reality''? Anyone?...Just me then...
The fact is that technology where both worlds can now physically interact with each other is amazing.Here is a clip of a remote controlled virtual avatar roaming and hiding behind a real world environment
The floor and building are both real surfaces, they just look virtual due to the compression of youtube. I love how the avatar is aware of edges and is able to hide behind object, games like Warhammer 40k could really benefit from this where they have very detailed game boards, characters could physically run and take cover behind the map.
To show the previous videos in a working scenario
This video shows a group of guys controlling some virtual vehicles that all interact with each other, real slopes and edges on the map. You can see them playing on an actual screen so although there is no game play other than drive, it still hosts lots of hope. Think of creating micro machine's levels out of items on your shelf, or rally race driving games. Even tabletop strategy games like R.U.S.E could take advantage of Augmented Reality.
When looking into how AR could implement into reality with recognition I found this
Basically the concept behind SixthSense is that Augmented Reality will fill in all the information you could want on a subject at that time, examples of this are projecting a watch on your write with a circular movement or book reviews and newspapers videos start upon changing pages. All of this information is displayed via a projector on the user's chest and coloured tabs on fingertips that can be identified. This technology could be advanced to work with a pair of glasses and able to be taken anywhere.
I found some other software that seemed very similar to SixthSense but in some ways more advanced, SixthSense needed coloured tabs on the fingers of the user to track and the projector didn't really show it tracking moving objects. The OmniTouch doesn't seem to share the same limitations.
The good thing about OmniTouch is it doesn't require coloured tabs on fingers and the projector can actually track quite quickly. Unfortunately the shoulder mounted projector isn't very discreet and looks as if it ache after a while of being worn.
A program similar to to SixthSense and OmniTouch is Layar.
Layar however is already realised and isn't as advanced what the previous videos would be, but again this is technology that is available now and not the near future.
Another interesting way of projecting an image is the DisplAir.
This is a really cool looking device, air being used to display an image is very interesting, even more so being able to interact with it. The problem however is that it can only be displayed to the size of the device and even then the quality isn't very good and will need some sort of black background to make it visible.
The Heliodisplay however have produced an advancement to this with their model L90.
As you can see the display is of a much higher quality and scale, also the Heliodisplay is able to visually produce it's display in 3D if playing a 3D file. This would be much more suited to augmented reality as the player would be able to interact with a much larger area.
This video shows how the use of mobile devices would progress AR.
Here it shows that information can be accessed directly on your phone via AR, from weather, businesses and travel times. This is all very possible and maybe only a few years until a reality.
This video also shows what else could come out of hand held devices in relation to AR.
At 2:10 the video shows how social networking could be used in AR, Facebook friend's faces could be displayed in real time with their status updates next to them. Examples of these apps like the Viewdle have already been tested and are said would exist when smartphones upgrade their processing power.
I believe that the future of AR will use some sort of head mounted display rather than a handheld devices, firstly because handheld's AR will be limited to the dimensions of the screen, and secondly the users arms will be constantly in use. Here are some modern HMD I looked at that could potentially be improved for specific AR functions.
Firstly the Vuzix WEAP 920
And the Sony HMZ-T1
Both of these devices would work great with AR as the player is able to have their hands free to use a controller. Also as the screen is over the user's eyes they will see the real world and virtual world as one. Without the need of looking at a screen with reality as a border, immersion will be a lot easier to achieve. The problem with these devices however are the prices and as long as they cost this much then they'll never reach mainstream.