Google Developers Blog: VR

New experimental features for Daydream

Friday, September 21, 2018

Posted by Jonathan Huang, Senior Product Manager, Google AR/VR Since we first launched Daydream, developers have responded by creating virtual reality (VR) experiences that are entertaining, educational and useful. Today, we're announcing a new set of experimental features for developers to use on the Lenovo Mirage Solo—our standalone Daydream headset—to continue to push the platform forward. Here's what's coming: Experimental 6DoF Controllers First, we're adding APIs to support positional controller tracking with six degrees of freedom—or 6DoF—to the Mirage Solo. With 6DoF tracking, you can move your hands more naturally in VR, just like you would in the physical world. To date, this type of experience has been limited to expensive PC-based VR with external tracking. We've also created experimental 6DoF controllers that use a unique optical tracking system to help developers start building with 6DoF features on the Mirage Solo. Instead of using expensive external cameras and sensors that have to be carefully calibrated, our system uses machine learning and off-the-shelf parts to accurately estimate the 3D position and orientation of the controllers. We're excited about this approach because it can reduce the need for expensive hardware and make 6DoF experiences more accessible to more people. We've already put these experimental controllers in the hands of a few developers and we're excited for more developers to start testing them soon.

Experimental 6DoF controllers See-Through Mode We're also introducing what we call see-through mode, which gives you the ability to see what's right in front of you in the physical world while you're wearing your VR headset. See-through mode takes advantage of our WorldSense technology, which was built to provide accurate, low latency tracking. And, because the tracking cameras on the Mirage Solo are positioned at approximately eye-distance apart, you also get precise depth perception. The result is a see-through mode good enough to let you play ping pong with your headset on.

Playing ping pong with see-through mode on the Mirage Solo. The combination of see-through mode and the Mirage Solo's tracking technology also opens up the door for developers to blend the digital and physical worlds in new ways by building Augmented Reality (AR) prototypes. Imagine, for example, an interior designer being able to plan a new layout for a room by adding virtual chairs, tables and decorations on top of the actual space.

Experimental app using objects from Poly, see-through mode and 6DoF Controllers to design a space in our office. Smartphone Android Apps in VR Finally, we're introducing the capability to open any smartphone Android app on your Daydream device, so you can use your favorite games, tools and more in VR. For example, you can play the popular indie game Mini Metro on a virtual big screen, so you have more space to view and plan your own intricate public transit system.

Playing Mini Metro on a virtual big screen in VR. With support for Android Apps in VR, developers will be able to add Daydream VR support to their existing 2D applications without having to start from scratch. The Chrome team re-used the existing 2D interfaces for Chrome Browser Sync, settings and more to provide a feature-rich browsing experience in Daydream.

The Chrome app on Daydream uses the 2D settings within VR. Try These Features We've really loved building with these tools and can't wait to see what you do with them. See-through mode and Android Apps in VR will be available for all developers to try soon. If you're a developer in the U.S., click here to learn more and apply now for an experimental 6DoF controller developer kit.

Open Sourcing Seurat: bringing high-fidelity scenes to mobile VR

Friday, May 4, 2018

Posted by Manfred Ernst, Software Engineer Great VR experiences make you feel like you're really somewhere else. To create deeply immersive experiences, there are a lot of factors that need to come together: amazing graphics, spatialized audio, and the ability to move around and feel like the world is responding to you. Last year at I/O, we announced Seurat as a powerful tool to help developers and creators bring high-fidelity graphics to standalone VR headsets with full positional tracking, like the Lenovo Mirage Solo with Daydream. Seurat is a scene simplification technology designed to process very complex 3D scenes into a representation that renders efficiently on mobile hardware. Here's how ILMxLAB was able to use Seurat to bring an incredibly detailed 'Rogue One: A Star Wars Story' scene to a standalone VR experience. Today, we're open sourcing Seurat to the developer community. You can now use Seurat to bring visually stunning scenes to your own VR applications and have the flexibility to customize the tool for your own workflows. Behind the scenes - How Seurat works Seurat works by taking advantage of the fact that VR scenes are typically viewed from within a limited viewing region, and leverages this to optimize the geometry and textures in your scene. It takes RGBD images (color and depth) as input and generates a textured mesh, targeting a configurable number of triangles, texture size, and fill rate, to simplify scenes beyond what traditional methods can achieve.

To demonstrate what Seurat can do, here's a snippet from Blade Runner: Revelations, which launched today with the Lenovo Mirage Solo. Blade Runner: Revolution by Alcon Interactive and Seismic Games The Blade Runner universe is known for its stunning worlds, and in Revelations, you get to unravel a mystery around fugitive Replicants in the futuristic but gritty streets. To create the look and feel for Revelations, Seismic used Seurat to bring a scene of 46.6 million triangles down to only 307,000, improving performance by more than 100x with almost no loss in visual quality: Original scene:

Seurat-processed scene:

If you're interested in learning more about Seurat or trying it out yourself, visit the Seurat GitHub page to access the documentation and source code. We're looking forward to seeing what you build!

Open Sourcing Resonance Audio

Wednesday, March 14, 2018

Posted by Eric Mauskopf, Product Manager

Spatial audio adds to your sense of presence when you're in VR or AR, making it feel and sound, like you're surrounded by a virtual or augmented world. And regardless of the display hardware you're using, spatial audio makes it possible to hear sounds coming from all around you. Resonance Audio, our spatial audio SDK launched last year, enables developers to create more realistic VR and AR experiences on mobile and desktop. We've seen a number of exciting experiences emerge across a variety of platforms using our SDK. Recent examples include apps like Pixar's Coco VR for Gear VR, Disney's Star Wars^TM: Jedi Challenges AR app for Android and iOS, and Runaway's Flutter VR for Daydream, which all used Resonance Audio technology. To accelerate adoption of immersive audio technology and strengthen the developer community around it, we’re opening Resonance Audio to a community-driven development model. By creating an open source spatial audio project optimized for mobile and desktop computing, any platform or software development tool provider can easily integrate with Resonance Audio. More cross-platform and tooling support means more distribution opportunities for content creators, without the worry of investing in costly porting projects. What's Included in the Open Source Project As part of our open source project, we're providing a reference implementation of YouTube's Ambisonic-based spatial audio decoder, compatible with the same Ambisonics format (Ambix ACN/SN3D) used by others in the industry. Using our reference implementation, developers can easily render Ambisonic content in their VR media and other applications, while benefiting from Ambisonics open source, royalty-free model. The project also includes encoding, sound field manipulation and decoding techniques, as well as head related transfer functions (HRTFs) that we've used to achieve rich spatial audio that scales across a wide spectrum of device types and platforms. Lastly, we're making our entire library of highly optimized DSP classes and functions, open to all. This includes resamplers, convolvers, filters, delay lines and other DSP capabilities. Additionally, developers can now use Resonance Audio's brand new Spectral Reverb, an efficient, high quality, constant complexity reverb effect, in their own projects. We've open sourced Resonance Audio as a standalone library and associated engine plugins, VST plugin, tutorials, and examples with the Apache 2.0 license. Most importantly, this means Resonance Audio is yours, so you're free to use Resonance Audio in your projects, no matter where you work. And if you see something you'd like to improve, submit a GitHub pull request to be reviewed by the Resonance Audio project committers. While the engine plugins for Unity, Unreal, FMOD, and Wwise will remain open source, going forward they will be maintained by project committers from our partners, Unity, Epic, Firelight Technologies, and Audiokinetic, respectively. If you're interested in learning more about Resonance Audio, check out the documentation on our developer site. If you want to get more involved, visit our GitHub to access the source code, build the project, download the latest release, or even start contributing. We're looking forward to building the future of immersive audio with all of you. #fullImg { width:100%; margin: 10px 0 0px 0; padding: 0; }

Announcing ARCore 1.0 and new updates to Google Lens

Friday, February 23, 2018

Anuj Gosalia, Director of Engineering, AR With ARCore and Google Lens, we're working to make smartphone cameras smarter. ARCore enables developers to build apps that can understand your environment and place objects and information in it. Google Lens uses your camera to help make sense of what you see, whether that's automatically creating contact information from a business card before you lose it, or soon being able to identify the breed of a cute dog you saw in the park. At Mobile World Congress, we're launching ARCore 1.0 along with new support for developers, and we're releasing updates for Lens and rolling it out to more people.

ARCore, Google's augmented reality SDK for Android, is out of preview and launching as version 1.0. Developers can now publish AR apps to the Play Store, and it's a great time to start building. ARCore works on 100 million Android smartphones, and advanced AR capabilities are available on all of these devices. It works on 13 different models right now (Google's Pixel, Pixel XL, Pixel 2 and Pixel 2 XL; Samsung's Galaxy S8, S8+, Note8, S7 and S7 edge; LGE's V30 and V30+ (Android O only); ASUS's Zenfone AR; and OnePlus's OnePlus 5). And beyond those available today, we're partnering with many manufacturers to enable their upcoming devices this year, including Samsung, Huawei, LGE, Motorola, ASUS, Xiaomi, HMD/Nokia, ZTE, Sony Mobile, and Vivo. Making ARCore work on more devices is only part of the equation. We're bringing developers additional improvements and support to make their AR development process faster and easier. ARCore 1.0 features improved environmental understanding that enables users to place virtual assets on textured surfaces like posters, furniture, toy boxes, books, cans and more. Android Studio Beta now supports ARCore in the Emulator, so you can quickly test your app in a virtual environment right from your desktop.

Everyone should get to experience augmented reality, so we're working to bring it to people everywhere, including China. We'll be supporting ARCore in China on partner devices sold there— starting with Huawei, Xiaomi and Samsung—to enable them to distribute AR apps through their app stores. We've partnered with a few great developers to showcase how they're planning to use AR in their apps. Snapchat has created an immersive experience that invites you into a "portal"—in this case, FC Barcelona's legendary Camp Nou stadium. Visualize different room interiors inside your home with Sotheby's International Realty. See Porsche's Mission E Concept vehicle right in your driveway, and explore how it works. With OTTO AR, choose pieces from an exclusive set of furniture and place them, true to scale, in a room. Ghostbusters World, based on the film franchise, is coming soon. In China, place furniture and over 100,000 other pieces with Easyhome Homestyler, see items and place them in your home when you shop on JD.com, or play games from NetEase, Wargaming and Game Insight.

With Google Lens, your phone's camera can help you understand the world around you, and, we're expanding availability of the Google Lens preview. With Lens in Google Photos, when you take a picture, you can get more information about what's in your photo. In the coming weeks, Lens will be available to all Google Photos English-language users who have the latest version of the app on Android and iOS. Also over the coming weeks, English-language users on compatible flagship devices will get the camera-based Lens experience within the Google Assistant. We'll add support for more devices over time. And while it's still a preview, we've continued to make improvements to Google Lens. Since launch, we've added text selection features, the ability to create contacts and events from a photo in one tap, and—in the coming weeks—improved support for recognizing common animals and plants, like different dog breeds and flowers.

Smarter cameras will enable our smartphones to do more. With ARCore 1.0, developers can start building delightful and helpful AR experiences for them right now. And Lens, powered by AI and computer vision, makes it easier to search and take action on what you see. As these technologies continue to grow, we'll see more ways that they can help people have fun and get more done on their phones.

Getting Started with the Poly API

Thursday, November 30, 2017

Posted by Bruno Oliveira, Software Engineer As developers, we all know that having the right assets is crucial to the success of a 3D application, especially with AR and VR apps. Since we launched Poly a few weeks ago, many developers have been downloading and using Poly models in their apps and games. To make this process easier and more powerful, today we launched the Poly API, which allows applications to dynamically search and download 3D assets at both edit and run time. The API is REST-based, so it's inherently cross-platform. To help you make the API calls and convert the results into objects that you can display in your app, we provide several toolkits and samples for some common game engines and platforms. Even if your engine or platform isn't included in this list, remember that the API is based on HTTP, which means you can call it from virtually any device that's connected to the Internet. Here are some of the things the API allows you to do:

List assets, with many possible filters:

keyword
category ("Animals", "Technology", "Transportation", etc.)
asset type (Blocks, Tilt Brush, etc)
complexity (low, medium, high complexity)
curated (only curated assets or all assets)

Get a particular asset by ID

Get the user's own assets

Get the user's liked assets

Download assets. Formats vary by asset type (OBJ, GLTF1, GLTF2).

Download material files and textures for assets.

Get asset metadata (author, title, description, license, creation time, etc)

Fetch thumbnails for assets

Poly Toolkit for Unity Developers If you are using Unity, we offer Poly Toolkit for Unity, a plugin that includes all the necessary functionality to automatically wrap the API calls and download and convert assets, exposing it through a simple C# API. For example, you can fetch and import an asset into your scene at runtime with a single line of code: PolyApi.GetAsset(ASSET_ID, result => { PolyApi.Import(result.Value, PolyImportOptions.Default()); }); Poly Toolkit optionally also handles authentication for you, so that you can list the signed in user's own private assets, or the assets that the user has liked on the Poly website. In addition, Poly Toolkit for Unity also comes with an editor window, where you can search for and import assets from Poly into your Unity scene directly from the editor.

Poly Toolkit for Unreal Developers If you are using Unreal, we also offer Poly Toolkit for Unreal, which wraps the API and performs automatic download and conversion of OBJs and Blocks models from Poly. It allows you to query for assets and filter results, download assets and import assets as ready-to-use Unreal actors that you can use in your game.

Credit: Piano by Bruno OliveiraHow to use Poly API with Android, Web or iOS app Not using a game engine? No problem! If you are developing for Android, check out our Android sample code, which includes a basic sample with no external dependencies, and also a sample that shows how to use the Poly API in conjunction with ARCore. The samples include:

Asynchronous HTTP connections to the Poly API.

Asynchronous downloading of asset files.

Conversion of OBJ and MTL files to OpenGL-compatible VBOs and IBOs.

Examples of basic shaders.

Integration with ARCore (dynamically downloads an object from Poly and lets the user place it in the scene).

Credit: Cactus wren by Poly by Google If you are an iOS developer, we have two samples for you as well: one using SceneKit and one using ARKit, showing how to build an iOS app that downloads and imports models from Poly. This includes all the logic necessary to open an HTTP connection, make the API requests, parse the results, build the 3D objects from the data and place them on the scene. For web developers, we also offer a complete WebGL sample using Three.js, showing how to get and display a particular asset, or perform searches. There is also a sample showing how to import and display Tilt Brush sketches.

Credit: Forest by Alex "SAFFY" Safayan No matter what engine or platform you are using, we hope that the Poly API will help bring high quality assets to your app and help you increase engagement with your users! You can find more information about the Poly API and our toolkits and samples on our developers site. .blgimg1 img { width: 100%; border: 0; margin: 0; padding: 10px 0 10px 0 ; } .blgimg2 img { width: 100%; border: 0; margin: 0; padding: 10px 0 10px 0 ; } .blgimg3 img { width: 100%; border: 0; margin: 0; padding: 10px 0 10px 0 ; } .blgimg4 img { width: 100%; border: 0; margin: 0; padding: 10px 0 10px 0 ; }

Resonance Audio: Multi-platform spatial audio at scale

Monday, November 6, 2017

Posted by Eric Mauskopf, Product Manager

As humans, we rely on sound to guide us through our environment, help us communicate with others and connect us with what's happening around us. Whether walking along a busy city street or attending a packed music concert, we're able to hear hundreds of sounds coming from different directions. So when it comes to AR, VR, games and even 360 video, you need rich sound to create an engaging immersive experience that makes you feel like you're really there. Today, we're releasing a new spatial audio software development kit (SDK) called Resonance Audio. It's based on technology from Google's VR Audio SDK, and it works at scale across mobile and desktop platforms. Experience spatial audio in our Audio Factory VR app for Daydream and SteamVR Performance that scales on mobile and desktop Bringing rich, dynamic audio environments into your VR, AR, gaming, or video experiences without affecting performance can be challenging. There are often few CPU resources allocated for audio, especially on mobile, which can limit the number of simultaneous high-fidelity 3D sound sources for complex environments. The SDK uses highly optimized digital signal processing algorithms based on higher order Ambisonics to spatialize hundreds of simultaneous 3D sound sources, without compromising audio quality, even on mobile. We're also introducing a new feature in Unity for precomputing highly realistic reverb effects that accurately match the acoustic properties of the environment, reducing CPU usage significantly during playback.

Using geometry-based reverb by assigning acoustic materials to a cathedral in Unity Multi-platform support for developers and sound designers We know how important it is that audio solutions integrate seamlessly with your preferred audio middleware and sound design tools. With Resonance Audio, we've released cross-platform SDKs for the most popular game engines, audio engines, and digital audio workstations (DAW) to streamline workflows, so you can focus on creating more immersive audio. The SDKs run on Android, iOS, Windows, MacOS and Linux platforms and provide integrations for Unity, Unreal Engine, FMOD, Wwise and DAWs. We also provide native APIs for C/C++, Java, Objective-C and the web. This multi-platform support enables developers to implement sound designs once, and easily deploy their project with consistent sounding results across the top mobile and desktop platforms. Sound designers can save time by using our new DAW plugin for accurately monitoring spatial audio that's destined for YouTube videos or apps developed with Resonance Audio SDKs. Web developers get the open source Resonance Audio Web SDK that works in the top web browsers by using the Web Audio API.

DAW plugin for sound designers to monitor audio destined for YouTube 360 videos or apps developed with the SDK Model complex Sound Environments Cutting edge features By providing powerful tools for accurately modeling complex sound environments, Resonance Audio goes beyond basic 3D spatialization. The SDK enables developers to control the direction acoustic waves propagate from sound sources. For example, when standing behind a guitar player, it can sound quieter than when standing in front. And when facing the direction of the guitar, it can sound louder than when your back is turned.

Controlling sound wave directivity for an acoustic guitar using the SDK Another SDK feature is automatically rendering near-field effects when sound sources get close to a listener's head, providing an accurate perception of distance, even when sources are close to the ear. The SDK also enables sound source spread, by specifying the width of the source, allowing sound to be simulated from a tiny point in space up to a wall of sound. We've also released an Ambisonic recording tool to spatially capture your sound design directly within Unity, save it to a file, and use it anywhere Ambisonic soundfield playback is supported, from game engines to YouTube videos. If you're interested in creating rich, immersive soundscapes using cutting-edge spatial audio technology, check out the Resonance Audio documentation on our developer site, let us know what you think through GitHub, and show us what you build with #ResonanceAudio on social media; we'll be resharing our favorites. .blogimg1 img { width: 100%; border: 0; margin: 0; padding: 20px 0 10px 0; } .blogimg2 img { width: 100%; border: 0; margin: 0; padding: 20px 0 10px 0; } .blogimg3 img { width: 100%; border: 0; margin: 0; padding: 20px 0 10px 0; }

Bringing Real-time Spatial Audio to the Web with Songbird

Friday, September 1, 2017

Posted by Jamieson Brettle and Drew Allen, Chrome Media Team For a virtual scene to be truly immersive, stunning visuals need to be accompanied by true spatial audio to create a realistic and believable experience. Spatial audio tools allow developers to include sounds that can come from any direction, and that are associated in 3D space with audio sources, thus completely enveloping the user in 360-degree sound. Spatial audio helps draw the user into a scene and creates the illusion of entering an entirely new world. To make this possible, the Chrome Media team has created Songbird, an open source, spatial audio encoding engine that works in any web browser by using the Web Audio API. The Songbird library takes in any number of mono audio streams and allows developers to programmatically place them in 3D space around the user. Songbird allows you to create immersive soundscapes, realistically reproducing reflection and reverb for the space you describe. Sounds bounce off walls and reflect off materials just as they would in real-life, capturing truly 360-degree sound. Songbird creates an ambisonic soundfield that can then be rendered in real-time for use in your application. We've partnered with the Omnitone project, which we blogged about last year, to add higher-order ambisonic support to Omnitone's binaural renderer to produce far more accurate sounding audio than ever before. Songbird encapsulates Omnitone and with it, developers can now add interactive, full-sphere audio to any web based application. Songbird can scale to any order ambisonics, thereby creating a more realistic sound and higher performance than what is achievable through standard Web Audio API.

Songbird Audio Processing Diagram The implementation of Songbird is based on the Google spatial media specification. It expects mono input and outputs ambisonic (multichannel) ACN channel layout with SN3D normalization. Detailed documentation may be found here. As the web emerges as an important VR platform for delivering content, spatial audio will play a vital role in users' embrace of this new medium. Songbird and Omnitone are key tools in enabling spatial audio on the web platform and establishing it as a preeminent platform for compelling VR experiences. Combining these audio experiences with 3D JavaScript libraries like three.js gives a glimpse into the future on the web.

Demo combining spatial sound in 3D environment This project was made possible through close collaboration with Google's Daydream and Web Audio teams. This collaboration allowed us to deliver similar audio capabilities to the web as are available to developers creating Daydream applications. We look forward to seeing what people do with Songbird now that it's open source. Check out the code on GitHub and let us know what you think. Also available are a number of demos on creating full spherical audio with Songbird. .blogimg img { width: 100%; border: 0; margin: 0; padding: 10px 0 0 0; } .blogcptn { width: 100%; text-align: center !important; font-style: italic; font-weight: bold; font-size: 85%; }

Experimenting with VR Ad formats at Area 120

Wednesday, June 28, 2017

Posted by Aayush Upadhyay and Neel Rao, Area 120 At Area 120, Google's internal workshop for experimental ideas, we're working on early-stage projects and quickly iterate to test concepts. We heard from developers that they're looking at how to make money to fund their VR applications, so we started experimenting with what a native, mobile VR ad format might look like. Developers and users have told us they want to avoid disruptive, hard-to-implement ad experiences in VR. So our first idea for a potential format presents a cube to users, with the option to engage with it and then see a video ad. By tapping on the cube or gazing at it for a few seconds, the cube opens a video player where the user can watch, and then easily close, the video. Here's how it works:

Our work focuses on a few key principles - VR ad formats should be easy for developers to implement, native to VR, flexible enough to customize, and useful and non-intrusive for users. Our Area 120 team has seen some encouraging results with a few test partners, and would love to work with the developer community as this work evolves - across Cardboard (on Android and iOS), Daydream and Samsung Gear VR. If you're a VR developer (or want to be one) and are interested in testing this format with us, please fill out this form to apply for our early access program. We have an early-stage SDK available and you can get up and running easily. We're excited to continue experimenting with this format and hope you'll join us for the ride!

Learn by doing with the Udacity VR Developer Nanodegree

Tuesday, September 27, 2016

Posted by Nathan Martz, Product Manager, Google VR With Google Cardboard and Daydream, our Google VR team is working to bring virtual reality to everyone. In addition to making VR more accessible by using the smartphone in your pocket, we recently launched the Google VR SDK out of beta, with native integration for Unity and UE4, to help make it easier for more developers to join the fold. To further support and encourage new developers to build VR experiences, we’ve partnered with Udacity to create the VR Developer Nanodegree. Students will learn how to create 3D environments, define behaviors, and make VR experiences comfortable, immersive, and performant. Even with more than 50 million installs of Google Cardboard apps on Google Play, these are still the early days of VR. Students who complete the VR Developer Nanodegree learn by doing, and will graduate having completed a portfolio of VR experiences. Learn more and sign up to receive VR Developer Nanodegree program updates at https://www.udacity.com/vr

Google VR SDK graduates out of beta

Thursday, September 22, 2016

Posted by Nathan Martz, Product Manager, Google VR At Google I/O, we announced Daydream—Google's platform for high quality, mobile virtual reality—and released early developer resources to get the community started with building for Daydream. Since then, the team has been hard at work, listening to feedback and evolving these resources into a suite of powerful developer tools. Today, we are proud to announce that the Google VR SDK 1.0 with support for Daydream has graduated out of beta, and is now available on the Daydream developer site. Our updated SDK simplifies common VR development tasks so you can focus on building immersive, interactive mobile VR applications for Daydream-ready phones and headsets, and supports integrated asynchronous reprojection, high fidelity spatialized audio, and interactions using the Daydream controller.

To make it even easier to start developing with the Google VR SDK 1.0, we’ve partnered with Unity and Unreal so you can use the game engines and tools you’re already familiar with. We’ve also updated the site with full documentation, reference sample apps, and tutorials. Native Unity integration This release marks the debut of native Daydream integration in Unity, which enables Daydream developers to take full advantage of all of Unity’s optimizations in VR rendering. It also adds support for features like head tracking, deep linking, and easy Android manifest configuration. Many Daydream launch apps are already working with the newest integration features, and you can now download the new Unity binary here and the Daydream plugin here. Native UE4 integration We’ve made significant improvements to our UE4 native integration that will help developers build better production-quality Daydream apps. The latest version introduces Daydream controller support in the editor, a neck model, new rendering optimizations, and much more. UE4 developers can download the source here. Get started today While the first Daydream-ready phones and headset are coming this fall, you can start developing high-quality Daydream apps right now with the Google VR SDK 1.0 and the DIY developer kit. We’re also opening applications to our Daydream Access Program (DAP) so we can work closely with even more developers building great content for Daydream. Submit your Daydream app proposal to apply to be part of our DAP. When you create content for the Daydream platform, you know your apps will work seamlessly across every Daydream-ready phone and headset. Daydream is just getting started, and we’re looking forward to working together to help you build new immersive, interactive VR experiences. Stay tuned for more information about Daydream-ready phones and the Daydream headset and controller coming soon.

Daydream Labs: positive social experiences in VR

Tuesday, August 9, 2016

Posted by Robbie Tilton, UX Designer, Google VR

At Daydream Labs, we have experimented with social interactions in VR. Just like in real reality, people naturally want to share and connect with others in VR. As developers and designers, we are excited to build social experiences that are fun and easy to use—but it’s just as important to make it safe and comfortable for all involved. Over the last year, we’ve learned a few ways to nudge people towards positive social experiences. What can happen without clear social norms People are curious and will test the limits of your VR experience. For example, when some people join a multiplayer app or game, they might wonder if they can reach their hand through another player’s head or stand inside another avatar’s body. Even with good intentions, this can make other people feel unsafe or uncomfortable. For example, in a shopping experiment we built for the HTC Vive, two people could enter a virtual storefront and try on different hats, sunglasses, and accessories. There was no limit to how or where they could place a virtual accessory, so some people stuck hats on friends anywhere they would stick—like in front of their eyes. This had the unfortunate effect of blocking their vision. If they couldn’t remove the hat in front of their eyes with their controllers, they had no other recourse than to take off their headset and end their VR experience. Protecting user safety Everyone should feel safe and comfortable in VR. If we can anticipate the actions of others, then we may be able to discourage negative social behavior before it starts. For example, by designing personal space around each user, you can prevent other people from invading that personal space. We built an experiment around playing poker where we tried new ways to discourage trolling. If someone left their seat at the poker table, their environment desaturated to black and white and their avatar would disappear from the other player’s view. A glowing blue personal space bubble would guide the person back to their seat. We found it’s enough to prevent a player from approaching their opponents to steal chips or invade personal space.
Reward positive behavior If you want people to interact in positive ways—like high-fiving 🙌 —try giving them an incentive. In one experiment, we detected when two different avatars “touched” each other’s hands at high speed. This triggered a loud slapping sound and a fireworks animation. It sounds simple, but people loved it. Meanwhile, if you tried to do something more aggressive, like punching an avatar’s body, nothing would happen. You can guess which behavior people naturally preferred.

Daydream Labs: animating 3D objects in VR

Tuesday, June 28, 2016

Rob Jagnow, Software Engineer, Google VR

Whether you're playing a game or watching a video, VR lets you step inside a new world and become the hero of a story. But what if you want to tell a story of your own?

Producing immersive 3D animation can be difficult and expensive. It requires complex software to set keyframes with splined interpolation or costly motion capture setups to track how live actors move through a scene. Professional animators spend considerable effort to create sequences that look expressive and natural. At Daydream Labs, we've been experimenting with ways to reduce technical complexity and even add a greater sense of play when animating in VR. In one experiment we built, people could bring characters to life by picking up toys, moving them through space and time, and then replay the scene.
As we saw people play with the animation experiment we built, we noticed a few things: The need for complex metaphors goes away in VR: What can be complicated in 2D can be made intuitive in 3D. Instead of animating with graph editors or icons representing location, people could simply reach out, grab a virtual toy, and carry it through the scene. These simple animations had a handmade charm that conveyed a surprising degree of emotion. The learning curve drops to zero: People were already familiar with how to interact with real toys, so they jumped right in and got started telling their stories. They didn't need a lengthy tutorial, and they were able to modify their animations and even add new characters without any additional help. People react to virtual environments the same way they react to real ones: When people entered a playful VR environment, they understood it was safe space to play with the toys around them. They felt comfortable performing and speaking in funny voices. They took more risks knowing the virtual environment was designed for play. To create more intricate animations, we also built another experiment that let people independently animate the joints of a single character. It let you record your character’s movement as you separately animated the feet, hands, and head — just like you would with a puppet. VR allows us to rethink software and make certain use cases more natural and intuitive. While this kind of animation system won’t replace professional tools, it can allow anyone to tell their own stories. There are many examples of using VR for storytelling, especially with video and animation, and we’re excited to see new perspectives as more creators share their stories in VR.

Daydream Labs: VR plays well with others

Tuesday, June 7, 2016

Posted by Rob Jagnow, Software Engineer, Google VR

At Daydream Labs, we pair engineers with designers to rapidly prototype virtual reality concepts, and we’ve already started to share our learnings with the VR community. This week, we focus on social. In many of our experiments, we’ve found that being in VR with others amplifies and improves experiences in VR, as long as you take a few things into account. Here’s what we’ve learned so far:Simplicity can be powerful: Avatars (or the virtual representations of people in VR) can be simplified to just a floating head with googly eyes and still convey a surprising degree of emotion, intent, and number of social cues. Eyes give people a location to look to and speak towards, but they also increase face-to-face communication by making even basic avatars feel more human. When we combine this with hands and a spatially-located voice, it comes together to create a sense of shared presence.Connecting the real and the virtual: Even when someone is alone in VR, you can make them feel connected. For example, you can continue to carry a conversation even if you’re not in VR with them. Your voice can serve as a subtle reminder that they’re spanning two spaces—the real and the virtual. This asymmetric experience can be a fun way to help ground party games where one player is in VR but other players aren’t, like with charades or Pictionary.But when someone else joins that virtual world with them, we’ve seen time and time again that the real world melts away. For most multiplayer activities, this is ideal because it makes the experience incredibly engaging. Join the party: When you first start a VR experience with others, it can be tough to know where to begin. After all, it’s easier to join a party than to start one! Create shared goals for multi-player experiences. When you give people something to play with together, it can help them break the ice, allow them to make friends, and have more fun in VR.You think you know somebody: Lastly, people who know each other offline immediately notice stature or differences in a person’s height in VR. We can re-calibrate environments to play with height and scale values to build a VR world where everyone appears to be the same height. Or we can adjust display settings to make each person feel like they’re the tallest person in the room. Height is such a powerful social cue in the real world and we can tune these settings in VR to nudge people into having more friendly, prosocial interactions.If you’d like to learn more about Daydream Labs and what we’ve learned so far, check out our recent Lessons Learned from VR Prototyping talk at Google I/O.

Daydream Labs: exploring and sharing VR’s possibilities

Thursday, May 19, 2016

Posted by Andrey Doronichev, Group Product Manager, Google VRIn Daydream Labs, the Google VR team explores virtual reality’s possibilities and shares what we learn with the world. While it’s still early days, the VR community has already come a long way in understanding what works well in VR across hardware, software, video, and much more. But, part of what makes developing for VR so exciting is that there’s still so much more to discover.Apps are a big focus for Daydream Labs. In the past year, we’ve built more than 60 app experiments that test different use cases and interaction designs. To learn fast, we build two new app prototypes each week. Not all of our experiments are successful, but we learn something new with each one.For example, in one week we built a virtual drum kit that used HTC Vive controllers as drumsticks. The following week, when we were debating how to make typing in VR more natural and playful, we thought — “what if we made a keyboard out of tiny drums?” We were initially skeptical that drumsticks could be more efficient than direct hand interaction, but the result surprised us. Not only was typing with drumsticks faster than with a laser pointer, it was really fun! We even built a game that lets you track your words per minute (mine was 50 wpm!).Daydream Labs is just getting started. This post is the first in an ongoing series sharing what we’ve learned through our experiments so stay tuned for more! You can also see more of what we’ve learned about VR interactions, immersion, and social design by watching our Google I/O talks on the live stream.

VR at Google - Jump, Expeditions, and Daydream

Thursday, May 19, 2016

Posted by Nathan Martz, Product Manager, DaydreamTwo years ago at Google I/O, we introduced Google Cardboard, a simple and fun way to experience virtual reality on your smartphone. Since then, we've grown the Google VR family with Expeditions and Jump, and this week at Google I/O, we announced Daydream, a platform for high quality mobile virtual reality. Jump—in the hands of creators and more cameras on the wayWe announced Jump, cameras and software to make producing beautiful VR video simple, at I/O last year. Jump cameras are now in the hands of media partners such as Paramount Pictures, The New York Times, and Discovery Communications. Virtual reality production companies including WEVR, Vrse, The Secret Location, Surreal, Specular Theory, Panograma, and RYOT also have cameras in hand. We can't wait to see the wide variety of immersive videos these creators will share with a growing VR audience. To enable cameras in a range of shapes and sizes and price points. Today, the Jump ecosystem expands with two partnerships to build Jump cameras. First, we're working with Yi Technology on a rig based around their new 4K Action Cam, coming later this year.With Jump, we've also seen incredible interest from filmmakers. Of course when you're creating the best content you want the absolute highest quality, cinema-grade camera available. To help create this, we're collaborating with IMAX to develop a very high-end cinema-grade Jump camera.Expeditions—One year, one million studentsMore than one million students from over 11 countries have taken an Expedition since we introduced the Google Expeditions Pioneer Program last May. The program lets students take virtual reality trips to over 200 places including Buckingham Palace, underwater in the Great Barrier Reef—and in seventh grader Lance Teeselink’s case—Dubai’s Burj Khalifa, the tallest building in the world. And soon, students will have even more places to visit, virtually, thanks to new partnerships with the Associated Press and Getty Images. These partners will provide the Expeditions program with high-resolution VR imagery for current events to help students better understand what’s happening around the world.Daydream—high quality VR on your Android smartphoneDaydream is our new platform for high quality mobile virtual reality, coming this fall. Over time, Daydream will encompass VR devices in many shapes and sizes, and Daydream will enable high quality VR on Android smartphones.We are working with a number of smartphone manufacturers to create a specification for Daydream-ready phones. These smartphones enable VR experiences with high-performance sensors for smooth, accurate head tracking, fast response displays to minimize blur, and powerful mobile processors. Daydream-ready phones take advantage of VR mode in Android N, a set of powerful optimizations for virtual reality built right into Android.With Daydream, we've also created a reference design for a comfortable headset and an intuitive controller. And, yes we're building one too. The headset and controller work in tandem to provide rich, immersive experiences. Take a look at how the controller lets you interact in VR:Build for DaydreamThe most important part of virtual reality is what you experience. Some of the world's best content creators and game studios are bringing their content to Daydream. You will also have your favorite Google apps including Play Movies, Street View, Google Photos, and YouTube. You can start building for Daydream today. The Google VR SDK now includes a C++ NDK. And if you develop with Unreal or Unity, Daydream will be natively supported by both engines. Visit the Daydream developer site where you can get access the tools. Plus, with Android N Developer Preview 3 you can use the Nexus 6P as a Daydream developer kit.This is just the beginning for Daydream. We’ll be sharing much more on this blog over the coming months. We’re excited to build the next chapter of VR with you.

Experience virtual reality art in your browser

Tuesday, April 19, 2016

Posted by Jeff Nusz, Data Arts Team, Pixel PainterTwo weeks ago, we introduced Tilt Brush, a new app that enables artists to use virtual reality to paint the 3D space around them. Part virtual reality, part physical reality, it can be difficult to describe how it feels without trying it firsthand. Today, we bring you a little closer to the experience of painting with Tilt Brush using the powers of the web in a new Chrome Experiment titled Virtual Art Sessions.

Virtual Art Sessions lets you observe six world-renowned artists as they develop blank canvases into beautiful works of art using Tilt Brush. Each session can be explored from start to finish from any angle, including the artist’s perspective – all viewable right from the browser. Participating artists include illustrator Christoph Niemann, fashion illustrator Katie Rodgers, sculptor Andrea Blasich, installation artist Seung Yul Oh, automotive concept designer Harald Belker, and street artist duo Sheryo & Yok. The artists’ unique approaches to this new medium become apparent when seeing them work inside their Tilt Brush creations. Watch this behind-the-scenes video to hear what the artists had to say about their experience:Virtual Art Sessions makes use of Google Chrome’s V8 Javascript engine for high-performance processing power to render large volumes of data in real time. This includes point cloud data of the artist’s physical form, 3D geometry data of the artwork, and position data of the VR controllers. It also relies on Chrome’s support of WebM video and WebGL to produce the 360° representations of the artists and artwork – the artist portrayals alone require the browser to draw over 200,000 points at 30 times a second. For a deeper look, read the technical case study or browse the project code that is available open source from the site’s tech page.We hope this experiment provides a window into the world of painting in virtual reality using Tilt Brush. We are excited by this new medium and hope the experience leaves you feeling the same. Visit g.co/VirtualArtSessions to start exploring.