Virtual Reality meets marine science in 360⁰

This is a 360° video. Use you mouse or device to look around the scene. This video is also available at Vrideo and 360Heroes

Harnessing technology popularised by the computer gaming industry, AIMS scientists are jumping in to explore the immersive technology of 360° video as a supplementary tool for underwater video survey methods.

This exciting new video technique removes the limits of the traditional video frame, allowing viewers to explore an entire recorded scene as if they were there. While the most popular uptake of this exciting platform has been within the entertainment and gaming industries, the technology has enormous potential within marine science with a number of projects, such as the Catlin surveys, already recording 360° still images.

Over the past 30 years, AIMS has collaborated in pioneering video survey techniques that are now used all over the world. Current methods such as the baited remote underwater video (BRUVS) and towed video methods are a valuable way of capturing and archiving information from the field and are used regularly for AIMS coral reef health monitoring, benthic habitat mapping and population studies.

Video has a number of advantages over human observers. Cameras can journey far deeper for longer periods than divers, and the information is consistent regardless of camera or conditions. The footage can also be archived for future use and be accessed by a range of expertise.

However, unlike divers, they are restricted to a relatively limited field of view. What happens beyond the frame of the camera is largely unknown. Using this new approach, researchers will have entire panoramas providing contextual information that can be stored and revisited for further analyses months or years later.

“360° video provides us with a greater capacity to determine ‘cause and effect’ from our marine survey videos” explains AIMS Visual Systems Researcher Scott Bainbridge. “Using our current video methods, we can see, for example, that a particular coral on the transect is stony white, but we can’t see what might have caused that. Using this new technique, we can potentially see the Crown-of-thorns starfish nearby that has eaten that coral. 360° video recreates what a diver can do; to be able to both focus on a small area but still be aware of the surrounding environment. The potential for unlocking this extra contextual information is incredibly exciting”.

In the future, this platform can be seamlessly integrated with current high-resolution techniques using data fusion. With 360° video providing the overall context, researchers can “drill down” into features or creatures of interest with ever-increasing detail, all from the one interface.

An AIMS diver carries a 6-camera, 360° video rig on the Great Barrier Reef

The immersive video platform is currently created from six, wide-angle action cameras held in a specially designed rig to capture an entire sphere of vision. The videos are then stitched together using dedicated software.

Viewers can choose to watch the videos in a number of ways. "The full immersive power of the technology is utilised by watching the videos on a dedicated headset, which gives the viewer the impression of being physically present during the scene and in the science" says Patrick Buerger, AIMS@JCU PhD student and 360° video science filmmaker. “This immersive experience is a great science communication tool, and allows the public, our stakeholders and other researchers to become involved in our research activities. Using the headset, viewers turn their head to discover another part of the scene. They can also view the video on their mobile phone, or on a computer, using the mouse to ‘spin’ their viewpoint.

A student dives virtually on the Great Barrier Reef at a recent discovery day at AIMS in Townsville. Off-the-shelf headsets provide a fantastic, immersive experience for 360° videos.

Still in its infancy, AIMS scientists are discovering how to best incorporate the platform with current techniques, including the current video-less manta tow method developed for our Long-term Monitoring Program. Scientists are also solving how to efficiently analyse and manage the large volume of data and files captured by this technique.