How To Do Augmented Reality On Iphone 10

Picture this: It's rainy Saturday afternoon and you're spending your day indoors at a local art museum. You meander from room to room staring at all the art, but not really absorbing any information. The information next to the art is too small, too crowded, or frankly, too boring. You quickly lose interest and make your way to the cafe.

If you're anything like me, the situation I just described sounds pretty familiar. Let's face it, it's hard to stay engaged at a museum when the information provided isn't that engaging or easily accessible. But what if there was a different way to consume information about artwork? Imagine if you could receive interesting information about a painting simply by pointing your phone at it — creating a frictionless and delightful experience.

museum augmented reality

Source: Cuseum

Sound far-fetched? It's not. In fact, the technology is already in use. And not only is it used in museums, it's part of your favorite social media apps too. What helps you become a dog and get cultural information at the touch of a button? Augmented reality (AR) of course!

But while this technology is familiar in some settings, it's definitely not simple. Behind every fun filter is augmented reality, or AR, hard at work — altering and enhancing our reality in real-time. And AR can do much more than transform your face or recognize your favorite paintings. It allows people to interact with digital objects in their home, in stores, and in public spaces.

We understand that augmented reality (AR) can be confusing. That's why we created this AR guide. Here we'll walk through what AR is, how it differs from virtual reality, its applications, and how growing businesses should use it.

What Is Augmented Reality (AR)?
Augmented Reality (AR) Applications and Examples
How SMBs Should Use Augmented Reality (AR)

What is augmented reality?

In simple terms, augmented reality (AR) is an embellished or altered form of reality where content lays over users' real-world views. The technology allows people to add digital assets to their physical environment. AR has a wide variety of uses — from assisting pilots and surgeons with complicated tasks to improving our Instagram stories with fun filters.

History of Augmented Reality (AR)

You're probably thinking, I guess I've used AR, but can we do a deeper dive? As we stated above, augmented reality (AR) is an enhanced or altered form of reality where superimposed content gets added to users' real-world views.

Those Snapchat filters? Yeah, that's augmented reality. Pokemon Go? Totally augmented reality. Oculus Rift? Well, no. That's actually virtual reality, and we'll get to that later. Augmented reality (AR) assists fighter pilots flying at nearly twice the speed of sound and helps surgeons to perform complicated procedures, but it wasn't always this advanced or accessible.

AR technology was born at Harvard University in 1968. Ivan Sutherland, an electrical engineering professor, created a head-mounted display system, nicknamed "The Sword of Damocles." Sounds intimidating, right? It was. The massive headset weighed so much that it was anchored to the ceiling to operate. Users also had to be strapped into the system for it to work, making the experience pretty uncomfortable.

Over the next several decades, advancements in AR led to helpful aviation, military, and industrial simulation tools, but the technology didn't gain a national audience until the late 1990s. One of the first widely visible uses of augmented reality came from an unexpected source: the NFL. The yellow line signifying a first-down, the one we have all grown to depend on over the past 20 years, is probably one of the most visible and helpful uses of AR.

Since then, AR has evolved at a rapid pace and is being used for both commercial and individual purposes. Between 2011 and 2013, AR was embraced by companies like Disney, Coca-Cola, and National Geographic to execute campaigns at large events and in public spaces like shopping malls and Times Square. In 2014, Google released Google Glass — the first mass-produced, wearable AR device — making it easy to get digital information simply by nodding your head. Snapchat added the geofilter feature a few months later, allowing users to add graphics showcasing geographic locations to their photos. They then introduced Lenses, a feature that maps users' faces to add motion graphics to photos and videos. As of the end of 2017, 187 million people used Snapchat daily. And that's just Snapchat. AR is now so popular that multiple social networks, businesses, and retailers use the technology. That's a lot of augmented reality.

Cameras and Sensors

To create augmented reality, you first need to capture some actual reality with sensors and cameras that gather information on the users' actual surroundings. This real-time information is a backdrop for the experience. Smartphone applications simply use your phone's built-in camera, while more complicated devices like Microsoft's HoloLens use a variety of specialized built-in cameras. In general, AR experiences work better with cameras that can read images in 3D, like the iPhone X's TrueDepth camera, since the depth information allows for more realistic experiences.

Processing

Realistic augmented reality also requires enough processing power to analyze inputs like acceleration, position, tilt, and depth in real-time to create immersive interactions. Fortunately for us, this is something our smartphones are now capable of doing without additional hardware. For this reason, we no longer need to mount our AR devices to the ceiling like the Sword of Damocles. But it wasn't easy getting to this point. It took Google years to shrink the three cameras and spacial awareness sensors to a size small enough to fit into a phone. As AR becomes more advanced, more devices will continue to incorporate the impressive technology.

Projection

After capturing real-world information, the augmented reality device then uses projection to layer digital renderings onto the scene. Currently, the projections display onto a smartphone screen or multiple screens within a wearable device. It's also possible to project directly onto surfaces, removing the need for any headset or screen at all.

Types of Augmented Reality (AR)

Although all AR devices share a few things in common, there are actually many kinds of augmented reality, and each one is better suited for different uses. In this section, we'll quickly go over five different types of AR and some of their strengths and weaknesses.

Marker-based, or image recognition augmented reality, uses a trigger object as a cue to display content. The trigger could be something like a QR code or even a cereal box. This type of AR requires the least amount of processing power and is fairly easy to implement, but it isn't as versatile as other kinds of AR because it relies on specific triggers being present.

Markerless augmented reality is more versatile than marker-based AR. Instead of trigger objects, this type of AR uses cameras, GPS, and accelerometer information to track where the user is and display relevant information. This combination of inputs is known as Simultaneous Localization And Mapping, or SLAM for short. Most types of AR available today use SLAM for markerless experiences.

As the name implies, projection-based AR projects digital images directly onto objects or surfaces within the user's environment. With projection-based AR, you could project a functioning keyboard on your desk. This type of AR negates the need for a screen or headset and allows users to create surreal experiences for large audiences. While impressive, projection AR isn't always the most practical option for smaller scale uses.

Outlining AR is also fairly self-explanatory. Instead of changing an entire scene, this type of AR uses image recognition to outline boundaries and shapes. It is most commonly used to help drivers see the edges of the road in low-light and to guide pilots towards landing strips. If the Titanic was setting sail now, they could have used outlining AR to avoid that iceberg … but then Leo and Kate would never have wowed us with their moving performances.

Superimposition based AR uses object recognition to partially or entirely replace an object within the user's environment with a digital image. For example, a doctor can use this type of augmented reality to add a digital x-ray over part of a patient's body during an operation.

Augmented Reality (AR) vs. Virtual Reality (VR)

We've covered some ins and outs of augmented reality, but you may still be thinking, "all of this still sounds a lot like virtual reality." So what's the difference? For starters, VR typically gets most of the attention from high-profile products like the Oculus Rift and Samsung Gear VR headsets. Virtual reality also goes one step beyond AR to create entirely new digital worlds.

When using VR, what you see and experience is different from what's actually around you. What you see and hear is entirely simulated. While this freedom creates lots of exciting opportunities, it also makes VR impractical for many common tasks and means you need to be careful when using headsets to avoid embarrassing situations. AR is less intrusive and easier to apply to everyday life since it combines added digital elements with the physical world around you.

Augmented Reality (AR) Applications and Examples

As you may have guessed, augmented reality has many uses beyond just digitally imposing flower crowns on your head or catching Pokemon. Because the technology is so adaptable, you can use AR just about everything. Here we'll talk about some of the more popular applications for augmented reality and provide some examples of it in use.

Transportation

As a frequent flier, I don't look for too much in my airlines. Just the basics like quality food, complimentary drinks, in-flight movies, a checked bag or two, free Wi-Fi, live TV, priority lounges, and pilots who can take off and land the plane. Augmented reality is helping with at least one of those things. Companies like Aero Glass have created augmented reality headsets that display airports, cities, navigation points, terrain features, other aircraft, and landing approaches for pilots. These features help pilots operate their planes, even when clouds or fog reduce visibility, which keeps flights safe and on time. That's something I think we can all get behind.

And air travel isn't the only mode of transportation AR is helping. If you're like most Americans, you likely spend around 293 hours, or a little over 12 days, driving each year. Tools like WayRay's Navion are changing the way we drive by projecting navigation instructions onto the windshield of the car. Navion also introduces gesture control commands to prevent drivers from looking down at their phones to enter or change a route. In addition to more intuitive navigation, these kinds of AR integrations have the potential to make roads safer by reducing the amount of time drivers spend looking away from the road.

Retail

While augmented reality can't assemble your IKEA furniture for you, it can help you decide which Ypperlig or Ekedalen table would look best in your dining room. With IKEA's new "IKEA Place" app, customers can preview over 2,000 pieces of virtual furniture in actual rooms within their home. This "try before you buy" model isn't limited to Scandinavian furniture stores — architects and engineers are also using augmented reality to sample building materials, finishes, and layouts before committing to a direction.

And you don't need to go into a store to try on makeup anymore. Sephora's Virtual Artist app allows users to try a variety of eye, lip, and cheek makeup by digitally adding it to an uploaded photo. The app also has pre-generated looks created by Sephora makeup artists and interactive tutorials that show how to use different makeup products. Apps like Virtual Artist remove barriers for consumers and help provide a clear path to purchase.

AR-Retail

Education

Augmented reality also has the potential to enhance education and learning. AR can transform textbooks and classrooms by turning previously static charts and images into interactive experiences. Geology suddenly sounds a lot more engaging when you can take apart the layers of a volcano or dive hundreds of miles beneath the Earth's crust using augmented reality.

Even flashcards, one of the simplest studying tools, can be improved with AR. Apps like AR Flashcards Animal Alphabet help young children learn the alphabet by bringing their flashcards to life. The ABCs sound like a lot more fun when the penguin from the "P is for penguin" card is standing in front of you.

Entertainment and Sports

Augmented reality is even changing the way we buy tickets to the Super Bowl. For Super Bowl LII, StubHub rolled out a feature on their mobile app that allowed ticket buyers to see a virtual 3D model of the U.S. Bank Stadium as well as the surrounding area. This wasn't the first time the ticket exchange company has experimented with AR. Previously, StubHub introduced "virtual view," which allowed users to see a preview of the view from their seats before they bought a ticket. After launching that feature, StubHub saw engagement double in a year.

Major sporting leagues have also embraced augmented reality as a way to enhance the viewing experience for their fans. The MLB's popular "At Bat" app plans on adding AR features this season that will allow users to see statistics on each player, ball velocity and distance traveled, and other information in real-time simply by pointing their phone at the field.

Marketing

Augmented reality's ability to create unique, immersive experiences makes the technology an excellent tool for marketers. Companies like IKEA, TopShop, and Converse use AR to allow customers to "try" their products before purchasing. These digital trial runs make sampling significantly easier and faster for shoppers, which can lead to more sales.

Even advertisements are made using AR. Many popular brands used AR in public spaces to delight viewers and grab their attention. In 2014, Pepsi installed outward facing cameras in a London bus shelter and used a live feed to project UFOs, giant robots, balloons, and a tiger on the loose inside the shelter. The experience made it look as if those scenes were actually happening on the street. The creative use of AR paid off for Pepsi. A YouTube video of the installation topped 6 million views, making it one of the most watched ad campaigns on YouTube at the time.

Healthcare

Some of the most promising applications for AR are within the healthcare industry. Today, medical students and doctors are using AR to learn or practice medical procedures. But AR's usefulness isn't just limited to life-threatening situations. AccuVein, a New York-based company, uses AR to help nurses find veins more easily when inserting IVs. This makes nurses' and patients' lives easier, increasing successful IV applications by 350%.

AR is also helping some patients with their recovery process. One company, called NuEyes, uses special AR glasses to help people with severe vision impairment. With the technology, NuEyes can help legally blind children see well enough to read and recognize their classmates.

There's even evidence that AR can help reduce excruciating phantom limb pain felt by amputees. By projecting a digital limb on to the patient, researchers were able to trick their brain into thinking the amputated limb was still there. This projection, paired with electrodes, allowed patients to practice relaxing the digital limb to ease their pain.

How SMBs Should Use Augmented Reality (AR)

With new AR developer platforms from Apple and Google, enterprise companies are beginning to experiment with the technology. Like I mentioned above, IKEA and Wayfair are allowing customers to place furniture in their homes without ever making an order, The New York Times is experimenting with AR news stories, and Starbucks is opening an immersive "coffee wonderland." But SMBs can also benefit from AR without writing a single line of code. According to HubSpot Research, businesses who worked with Pokemon Go to make their storefronts into PokeSpots saw a $2,000 average increase in weekly sales due to additional foot traffic. Talk about a growth opportunity! Growing businesses should make it a practice to look out for similar ways to inexpensively partner with existing AR experiences.

If you'd like to create an AR experience of your own, we recommend starting with your customer journey and working outwards from there. Any company can build an AR application, but not all of them will provide value to their customers. AR should make it easier for your customers to interact with your brand in a meaningful way that drives them to purchase. For example, if you sell physical products, let your consumers imagine or try them at home.

If you're trying to decide between prioritizing AR or VR, we recommend AR. VR requires expensive, unique operating systems that only a small portion of the population has access to through expensive headsets, while the majority of people have an AR device right in their pocket … a cell phone. For businesses with a great idea and technical abilities, adopting AR early could pay off in a big way.

Conclusion

While augmented reality has been around for several decades, we are only just learning about and experiencing its true potential. AR's ability to connect both the physical and digital worlds makes it adaptable for many use cases. The technology's numerous adaptations are helping to increase our productivity, standard of living, and quality of entertainment.

Businesses with the right development capabilities and content ideas should consider how AR could help improve both their business operations and their customers' experience. Adoption of AR technology may have gotten off to a slow start, but with new developer platforms, there's no telling how popular this technology could be.

Originally published Oct 10, 2018 8:00:00 AM, updated September 30 2020