A new update to Oculus’ Quest 2 virtual reality system will let developers incorporate real-life video from the VR headset’s sensors into their games to create a mixed reality experience.
The API is called Passthrough API and is currently experimental. With the API, developers can customize how a player’s surroundings appear through their VR headset while applying effects and filters and even rendering the real world onto certain in-game surfaces.
The feature is first rolling out to Unity developers in an upcoming software update, “with support for other development platforms coming in the future,” Oculus VR said in a blog post on Friday.
The API gives you the ability to toggle the mixed reality’s opacity. This essentially means that you can control how much of the real world or virtual one you can see at any given time.
Other than games, Oculus also showed how it could improve productivity and enable a more collaborative workspace by incorporating workers’ real-life keyboards and desks. Users will be able to interact with content that is virtual without having to lose sight of what’s real besides them.
“Apps that use Passthrough API cannot access, view, or store images or videos of your physical environment from the Oculus Quest 2 sensors. This means raw images from device sensors are processed on-device,” the company said.