Deployment and Evaluation of a 5G Open Spatial Computing Platform in a Dense Urban Environment

---

Implementation plan :

The common vision of a “real-world metaverse” is to be able to place 3D digital information linked to any physical place, enhancing our ability to gather information at a glance. A fundamental condition to the creation of persistent AR experiences is to capture and create a detailed 3D map of the environment. In existing AR cloud solutions today, the images and 3D maps of a user's personal environment are centralised to a few industry giants. Apart from not only storing very intimate data about users' surroundings, all the digital information displayed in AR is also hosted and filtered (“personalized”) by these providers. To address these privacy concerns, this project aims to advance and make publicly available all components of an end-to-end AR cloud solution, the Open Spatial Computing Platform (OSCP). We start with creating a new reference client in Unity (for interoperability) and then implement two representative AR cloud use cases. We also adapt the existing components to be deployable at new locations, and we demonstrate the complete system in action in the COSMOS 5G testbed in Manhattan. Prior to this project, AR contents were added manually to the content database which was not user friendly. We therefore started the first phase of this project with developing a new client application with the ability to easily place content directly in augmented reality. Our goal is that any client that adheres to the OSCP protocols will be able to place new contents, which will be immediately discoverable by other clients (independently from their underlying device type, operating system, or AR engine). Related to content placement, we also had to implement user authentication and an open-source file storage solution that anyone can easily set up (note that the OSCP does not store but only references contents).

In the second phase, we extend our client application with new features required by the defined use cases. In the first representative use case, a mobile device discovers a sensor node in a laboratory environment and shows its current sensor values floating above the real node. Beyond the discovery, this requires real-time streaming of IoT data from sensors to AR devices. In the second representative use case, our plan is to have an AR art piece placed at a specific location or implement a collaborative drawing application. This use case requires real-time sharing of users' poses and interactions with each other. The project also requires creating a 3D map of the selected experiment locations, as well as creating simplified digital twins of the locations to aid the artistic design of AR experiences. The 3D map was built by the US and EU partners jointly with two different technologies. The digital twin is created by the EU team using a series of photos taken by the US team at the Rutgers University lab. We will publish a detailed process description of generating a 3D map to be used as a reference in this project.

Impacts :

• Enhanced EU – US cooperation in Next Generation Internet, including policy cooperation.
• Reinforced collaboration and increased synergies between the Next Generation Internet and the Tomorrow's Internet programmes.
• Developing interoperable solutions and joint demonstrators, contributions to standards.
• An EU - US ecosystem of top researchers, hi-tech start-ups / SMEs and Internetrelated communities collaborating on the evolution of the Internet.

Results :

• Installation and configuration of all OSCP components on US testbed for use by the research team on the COSMOS network on the COSMOS network
• Heterogeneous urban environment of campus scale captured in 3D for testing with AR experiences and ready for development of future vision based localization services
• Measurement of AR over wireless network within expected performance parameters
• Two AR applications implemented on top of the Open Spatial Computing Platform
• Discovery and filtering of AR assets for experiences using Spatial Content Discovery
• Discovery and filtering of AR assets for experiences using Spatial Content Discovery

Future Plan :

The innovative elements introduced in this project will include the seamless detection of spatial context and (through the use of the GeoPose standard) automatically provide the user the best 3D map and most relevant content as the user change’s locations along a trajectory in a target rich environment including buildings and open spaces the project may further study aspects of identity, privacy and personal data protection. Efficient and continuous 3D world capture, mapping and query technologies may, due to their invasive nature, introduce risks to private and public institutions and rights of users and entities. Control over sensitive/private data, privacy and security in a diverse range of contexts such as personal, on campus, in public, on-premises, in restricted facilities and under emergency settings are high on the OARC’s roadmap for further developments.

The 5G testbed will gain a service that enables mobile camera localization and real-time location aware content discovery on top of the existing platforms. The OSCP also serves as the spatial context layer for location- based and personalized information retrieval in smart environments. These features can enable further sensing research and (in case of public testbeds) may also serve as an incentive service for a large number of users to join gamified experiments on the testbed.

As the backbone of location-based applications for students and campus visitors, the deployed platform can further serve (even after the project is finished) as a learning and test environment. We envision location- anchored and community- created digital media content, interactive visual navigation, restaurant discovery, lecture discovery, ride-hailing, etc. application to name only a few.

Key results

This project has successfully applied for further funding. Website for project Aurora  https://www.auroraviewer.org/auroraviewer

 

Tutorial videos:

• Open Source visual positioning Service:   https://youtu.be/ZKaBXdxD0Fs
• Aurora View, the Unity client app : https://youtu.be/qUR3zcTDuXA

 

NGI related Topic :

Discovery and identification technologies

Call Reference :

3