Augmented Reality, Virtual Reality, and Mixed Reality

Sascha L. Schmidt / Johannes Fühner - February 1, 2022

Tips for practitioners

In the summer of 2016, many witnessed an entirely new phenomenon as they walked through their city: People were wandering around busy streets – eyes glued to their phones – trying to spot a virtualPokémon in the real world. The first major success story of augmentedreality had been born. The game Pokémon Go celebrated its fifth birthday in June 2021, and it has racked up sales of approximately fivebillion US dollars and more than 600 million app downloads since.

In the meantime, augmented reality in Pokémon Go has become just one of the many examples for applying what are known as extended realities. To better understand the market, it is important to distinguish between the various terms first. Extended reality is a collective term that covers various technologies such as augmented reality (AR), virtual reality (VR), and mixed reality (MR).

Overall, the extended reality market is worth around 31billion US dollars today and is expected to increase almost tenfold to 297 billion US dollars by 2025. In terms of size, the market is thus on par with that of other key technologies such as robotics or artificialintelligence. Also in sports, the technology has taken hold and offers completely new kinds of fan experiences. With that in mind, this article will discuss selected use cases and provide an outlook on what it will all mean for the future of sports business.

Media use case: from the living room to the playing field

AR has been in established use in sports television broadcasts for a long time. Well-known examples include the "first-down line" in American football, the trajectory curve in golf, and distance lines in ski jumping. VR applications have likewise been around for several years. At the 2016 Olympic Games in Rio de Janeiro, VR broadcasts were produced on a large scale for the first time. As a result, Samsung users were able to enjoy almost 100 hours of live sports, provided they had a Gear VR headset, which had to be purchased first for around 100 euros.

However, the true pioneers of extended reality have always been gaming and eSports. According to a PwC study, gaming accounts for around one third of all VR sales in Germany. VR-enabled simulations are becoming increasingly popular in eSports as well. In 2018, for example, the Electronic Sports League (ESL), in collaboration with Oculus, founded a VR-specific competition series by the name of VR League, which currently provides contests in four different titles.

It is here that the VR experience offers not only gamers but also spectators an exciting opportunity to enjoy the game action up close. In September 2019, for instance, Sony filed a global patent for eSports events that allow spectators to participate in events using a social, interactive VR application. However – despite these pilot projects – it's still early days in terms of development. In soccer, too, many clubs and leagues have already begun to use extended reality to enhance the media experience. The partnership between Manchester City and Jaunt VR is one well-known example. Jaunt VR, which counts Disney and Axel Springer among its investors, developed a VR platform for Manchester City that allows fans from around the world to access the players' dressing room and to run out onto the playing field along with their stars. In 2018, 1. FC Köln and others in the Bundesliga experimented with an AR/VR app for the first time. Using this app, fans can visit a virtual museum or have the team lineup projected onto their screen on match day.

The German football league DFL has been collaborating with sports marketing company AIM and others to offer country-specific perimeter advertising using what are known as digital overlays. Moreover, DFL is testing various AR innovations in the AR realm. For example, a group of test subjects got to watch the 2021 Super Cup match between FC Bayern Munich and Borussia Dortmund as an AR projection through special glasses referred to as "smart glasses." The match was streamed live on a virtual TV screen inside the glasses. What’s more, various information items such as real-time statistics and game graphics were displayed as well as a "miniature playing field" so viewers could track the positions of all the players using 3D projection.

Users were also given a variety of interaction options, were able to call up on-demand content such as live statistics on all players – like a heat map or a player's current speed – and rewatch individual scenes in 3D. The application uses the ARISE platform developed by French startup Immersiv.io, a specialist in AR technology for sports. Immersiv.io had previously collaborated with DFL and Vodafone in 2019, helping them to develop a 5G stadium app.

Many of the above-mentioned innovations promoted by sports rights holders are still in their infancy. However, the quality of extended reality technologies can be expected to improve while becoming more cost-effective in the future. As the cost of hardware decreases, a variety of opportunities will emerge for sports organizations to strengthen fan engagement among global fan segments.

Stadium use case: from the stands to the playing field

The ever-improving options available for consuming a favorite sport from afar beg the rather delicate question: What type of live stadium experience will even be needed in the future? Yet one thing is sure: Event organizers face increasing pressure to make stadium attendance more attractive. Extended reality can play its part in helping them succeed.

Again, the pioneers in this area stem from US sports. In the NBA, Vodafone has launched an AR pilot where spectators can tap players on the field using their smartphones in order to have live statistics displayed. The NFL has also been busy experimenting around. The Minnesota Vikings have been harnessing the power of AR in their app for approximately four years. In 2019, the NFL team launched a collaboration with Pepsi, introducing a real-time AR game among stadium visitors. The Dallas Cowboys are another case in point. They provide an AR-powered photo booth in the stadium where visitors can snap a group picture with their personal favorite players. Dallas Cowboys sources have reported that this campaign generated over 50 million social media impressions.

Aside from AR, MR likewise offers promising potential for promoting new fan experiences in the stadium. Researchers at the University of Glasgow are currently working on holograms with haptic properties. As a stadium visitor, you may one day be able to use them to touch and move virtual objects or people. A first prototype debuted at the Carolina Panthers 2021 season opening, featuring a larger-than-life panther bounding through the real stadium.

Another future scenario is the construction of satellite stadiums. The idea behind this is to project the "real" sporting event live into a satellite stadium using 360-degree 3D video and VR or MR technology. It will allow fans who live far away from their favorite team to visit the stadium, too. Initial efforts attempting to make this a reality have already been mounted. One example is Caesars Sportsbook in Las Vegas, where visitors are treated to a stadium-like atmosphere with 143-inch LED screens. Fans at the Manchester United Experience Center in China get to visit Old Trafford virtually, also by means of VR technology.

At this point it is obviously too early to tell whether, in a time to come, satellite stadiums will be able to deliver the same emotionality as a live experience. Be that as it may, the analog and digital worlds will increasingly merge together. Technology experts from Silicon Valley such as Microsoft CEO Satya Nadella refer to this as the "metaverse", where real and virtual characters can interact with each other. Experts view the metaverse as the next evolution step of the internet which has received increasing attention after Facebook rebranding its new name "Meta". The gaming segment is certainly a metaverse pioneer. In April 2020, for example, US artist Travis Scott plunged into the virtual world of Fortnite, attracting more than twelve million live views with his live concert.

In sum, sports organizations will need to find innovative solutions to preserve the uniqueness of a stadium visit, especially since it is currently unclear how ticket demand will develop in the medium term after the COVID-19 pandemic. Extended reality can help boost the creation of novel fan experiences at the stadium.

Training use case: tapping a source of competitive advantage

Many recreational athletes will be familiar with this type of application from the gym: Modern fitness equipment allows you to immerse yourself into virtual worlds where you can take your run to a faraway city or climb a famous Tour de France mountain. At the very least since the rise of Peloton and Zwift, working out in your own living room has reached significant scale. In this context, Facebook CEO Mark Zuckerberg and others have asserted their belief on multiple occasions that extended reality is creating enormous potential in the sports industry.

In addition to recreational sports, extended reality may improve training opportunities in professional sports, as well. The first known use case happened over 20 years ago, when the US bobsled team used a VR simulator to prepare for the 1998 Winter Olympics in Nagano. A lot has happened since then, as some recent examples show:

In American football, the company Strivr has been causing a stir as it collaborates closely with a Stanford University team to offer VR-based training programs. Strivr is also testing similar approaches in basketball, where athletes no longer train exclusively on the court but also in an artificially generated environment, wearing VR headsets. For instance, players can train to focus their gaze on the essentials in order to make better decisions on the court later on.

European soccer has also begun to sit up and take notice, with companies like Rezzil providing VR-based training in the Premier League during the COVID-19 pandemic. The startup has succeeded in raising some two million pounds with seed funding. Investors include ex-pros Gary Neville and Vincent Kompany.

Interesting startups outside of ball sports include RideOn Vision, which is developing smart ski goggles that offer route suggestions and record track time on the slopes. Following successful pilot projects, the company is now preparing for serial production.

Currently, the added value in training is still limited to very specific fields of application. In the long term, however, this technology will be relevant for both top-level and mass sports. The great advantage of extended reality is that you can practice anywhere and anytime without needing a training partner.

Conclusion

Examples of the successful application of extended reality in sports already abound. If the technology continues to develop at its current pace, scalable business models will be just a matter of time. Decision makers in the sports business need to address the topic today, especially because it’s extremely relevant for young target groups.

Literature reference and methodology

• Detailed information on this and other topics can be found in the original publication: Schmidt, S. L. (2020): 21st Century Sports - How Technologies Will Change Sports in the Digital Age, Springer Cham, September 2020. Doi.org/10.1007/978-3-030-50801