Chapter 6: How Augmented Reality is Shaping Mobile App Development

The augmented reality industry is flourishing in popularity as the number of people using AR-based apps is expected to grow to 200 million users by 2020, which is a big jump compared to the 60 million users in 2013. Statista also predicts that the economic impact of the AR/VR markets will reach 29.5 billion U.S. dollars by 2020.

In a previous post about how AR is being used in different businesses and industries, we discussed the definition of augmented reality and how it differs from virtual reality. As a reminder to you, AR involves the integration of digital components in the physical world. It is an overlay of computer-generated content on our existing natural environment. This could be anything like graphics and sounds to enhance the perception of reality and boost the user experience.

The following chapter will delve into the different AR categories, the hardware involved, how it is impacting mobile app development, the types of AR development platforms for Google and Apple and some popular apps using the technology. Don’t forget to read the previous 5 chapters on an introduction to app development terminology, knowing when to choose hybrid or native apps, understanding the use of connected objects, a general glance at virtual reality and how artificial intelligence is changing the app industry.

What are the Current Existing Augmented Reality Categories?

As we just explained in the introduction, augmented reality is used in apps for mobile devices to add digital aspects to reality, essentially enhancing the real world and adding to what already exists. For instance, using a Snapchat filter that adds bunny ears or a dog tongue to your selfie. But did you know there are different categories to choose from when building an app with augmented reality features? The following are the 4 most well-known types:

Marker-based AR: This is also known as image recognition which uses a camera and black and white visual markers, like a QR code, to trigger augmented reality content. The camera must be pointed at the marker, and once the device recognizes it, the application will overlay the digital content on top. When creating a marker based AR app, the images or descriptors are provided beforehand. These objects are therefore hard-coded into the app, making them easier to detect when the camera is analyzing the marker.

Markerless AR: This is the most common category which is also known as location-based AR. These apps can recognize images and objects that weren’t hard-coded into the app beforehand. This means they can work without markers. They can detect the user’s position using a GPS and digital compass and then overlay digital objects in that location on top of the physical space. An example of this type would be the popular game Pokémon Go. These types of AR apps provide data based on your specific location.  

Projection Based AR: This involves projecting interactive artificial light on real physical objects and surfaces, for example, a digital keyboard projected on your desk. It could also include projecting 3D interactive holograms in the air.

Superimposition based AR: This type of AR app can replace an entire object or part of one with a new augmented reality view. An example of this type would be the Ikea augmented reality catalog. When you download the app, you can place the catalog anywhere in your home and it acts as an anchor to see 3D images of a piece of furniture like a couch or kitchen table you liked in the catalog.

Augmented Reality Platforms and Hardware

Augmented reality platforms such as Apple’s ARKit and Google’s ARCore allow for unique app creations where developers can add AR features to new or existing applications. With these platforms, iOS and Android developers are able to experiment with overlaying digital elements on top of the real world. Below we mention the features in both:

ARKit (Apple): There are 3 main important features including object detection and motion tracking the device’s position as the camera captures live objects, environmental understanding as the camera scans and analyses the scene (this could be on a coffee table, the ground or on walls), and light emission or estimation which measures and detects the presence of light in the area.

ARCore (Google): This platform has the same 3 main features as the ARKit but with some slight differences to each element. The motion tracking measures the shape and format of the surrounding objects to detect the orientation of the Android device. The environmental understanding of the device is similar to the ARKit as the ARCore also helps understand the device’s setting and environment. Finally, the light estimation feature detects the light in the surrounding area and distributes the balance of light.

Along with the various platforms for AR app creation, like with virtual reality, there are different types of hardware to accompany the technology to enhance the overall user experience. With augmented reality, heads-up displays (HUD) are used. These are transparent displays that present data without people having to look away from their regular and natural field of view. This hardware usually comes in the form of glasses, putting the AR experience directly in front of the user’s face.

There are also handheld devices like smartphones or tablets using AR apps which then put the augmented reality experience in the user’s hands. The difference from certain virtual reality hardware is that AR is completely untethered. There is no need for cables or desktop computers for the hardware to function. It often uses Bluetooth to connect the smartphone to the glasses, offering a completely hands-free experience. Some examples of well known augmented reality glasses include:

Vuzix Blade:
Features: 2.5-hour battery life, HD camera, mapping directions, view restaurant menus, receive weather information and alerts, provides users with the information from their smartphone hands-free.

Solos:
Features: Most often used for cycling and running, provides performance analytics, audio and visual navigational cues, can review performance improvements, save your favourite routes and tracks time and distance.

Sony SmartEyeGlass:
Features: Lightweight hardware, text, images and symbols are superimposed in your natural field of view, communicates with a smartphone over Bluetooth, uses hologram optics technology, sends navigation instructions, hands-free assembly instructions and retrieves sports statistics.

How Augmented Reality is Part of the Mobile App Development Process

As we mentioned in our previous post on the use of augmented reality in businesses and industries, the immense popularity of the Pokémon Go app released in 2016 is credited for having sparked the AR movement. After all, this app was downloaded over 800 million times as of March 2018. The game showed the incredible possibilities augmented reality has to offer mobile app development and how utilizing the technology is an effective way to engage users.

Adding the element of AR to applications adds another level of customer engagement and experience. With all the frenzy and excitement surrounding this technology, you are probably wondering how you can incorporate augmented reality into your app brand. Below we mention some general development phases to consider when moving forward with an AR project.

• Choose the AR elements to incorporate: After reading this post, you have a solid understanding of augmented reality. As we mentioned earlier on, augmented reality involves incorporating digital objects into the physical world. You have to ask yourself how you want to include AR into your app. This could be adding informational facts at different locations using the GPS or it could even be when the camera is pointed at a certain body part, the app could show the internal anatomy. It is up to you to decide what kind of AR functionality you want your app to have.  
• Know how you want to integrate AR elements: This means choosing the proper AR platform to develop the application, like the ARKit or ARCore depending on what device you plan on using.
• What will be the user’s interaction with the AR app? This includes if the users will receive push notifications, how they will launch the AR features in the app (voice activation, hitting a button etc…) and if they need some sort of anchor or QR code as a reference point for the augmented reality features to work.
• Choose an AR category: As we mentioned above, there are 4 different AR categories to choose from. It is up to you to choose between, marker, markerless/location, projection or superimposition based augmented reality. You must figure out the AR elements you wish to incorporate as well as knowing how you want your users to experience and engage with your app.
• Wearable support: Will your app allow for connections to augmented reality hardware like smart glasses? Do you want your users to have a hands-free experience? You must answer these questions and have a clear picture of what you are looking for before jumping into your augmented reality project.

Apps using Augmented Reality

Now that you are a mini-expert in augmented reality, we found some of the most popular apps utilizing the technology for you to try. Check it out!

Flirtar: Flirting in Augmented Reality
According to the team: “Flirtar is the world’s first dating app that uses Augmented Reality (AR) Using geolocation, we will connect you to fellow Flirtar users wherever you go or want to be. Meet people in the real world, in real time. Suppose you’re at a crowded place and suddenly you see someone that makes your heart skip a beat. Open up your Flirtar app using the AR experience, and if he or she is a registered user, we will show you their name, age, interests, depending on how that user has configured their profile and permissions. The person you would like to meet will get notified, and if they are interested, just start a conversation!”

Ingress: Turning Reality Into a Game
According to the team: “Ingress transforms the real world into the landscape for a global game of mystery, intrigue, and competition. Our future is at stake. You must choose a side. A mysterious energy has been unearthed by a team of scientists in Europe. The origin and purpose of this force is unknown, but some researchers believe it is influencing the way we think. We must control it or it will control us. “The Enlightened” seek to embrace the power that this energy may bestow upon us. “The Resistance” struggle to defend, and protect what’s left of our humanity. Install Ingress and transform your world.”

Quiver: Augmented Reality Colouring
According to the team: “Coloring pages have never been so much fun! The Quiver App combines physical coloring from “back in the day” with state of the art augmented reality technology to bring you and your children an extraordinarily magical experience. You can find our free pages by downloading the app and on our website. Save and print the pages directly from the Quiver App or from a computer to start enjoying the fun. By using our unmatchable expertise in augmented reality, the Quiver App enables children and adults to unleash their inner artist and interact with their personally customized creations.”

Zombie Go: Killing Zombies in Your Neighbourhood
According to the team: “Zombie GO™ – The Real World Zombie Game – An Augmented-Reality Shooter – Hunt Down Zombies as You Walk Around Your City, House, Job or Wherever. Survive an On Slaught of The Dead Popping Up Around Every Corner, Powerup and Take Them On One on One With Your Camera Phone Pistol.”

Sephora: Virtual Artist
According to the team: “Makeover on the go: Instantly try on lip, cheek & eye makeup. Try looks created by Sephora experts for your makeover. Makeup tips to suit your style from the Virtual Artist”

Just a Line: Draw in AR
According to the team: “Just a Line lets you make simple drawings in augmented reality, then share your creation with a short video. Draw on your own or with a friend, then hit record and share what you made with #justaline.”

Be sure to stay tuned for the final chapter in our Comprehensive Handbook on App Development for the Non-Tech Savvy where we take a look at mixed reality and how it incorporates both augmented and virtual reality elements. We hope you have enjoyed learning about the different components of app development so far and don’t forget to catch up on the previous 5 chapters!

Related Posts

Chapter 1: Intro Terminology
Chapter 2: Hybrid vs Native
Chapter 3: Connected Objects
Chapter 4: Virtual Reality
Chapter 5: Artificial Intelligence Industry