3D digital twin to enhance understanding of the Great Barrier Reef

Photo-realistic 3D images of sections of the Great Barrier Reef that will aid recovery and management efforts could be produced faster and more accurately thanks to a new partnership between the Australian Institute of Marine Science (AIMS) and La Trobe University.

Researchers from both organisations are testing the capabilities of AI software developed at La Trobe called Reef-NeRF (Neural Radiance Field) and Reef-3DGS (Gaussian Splatting) which enables real-time rendering of photo-realistic scenes, to produce 3D digital twins of the sections of the reef.

Currently AIMS uses photogrammetry to produce digital twins of sections of the Reef. The 3D images, processed through supercomputers, help scientists to study and measure its nooks and crannies across unprecedented spatial scales.

NeRF and 3DGS technology has already been applied in terrestrial settings with promising results. It produces more photo realistic digital twins, faster, but is unproven for underwater datasets or coral reef science. The researchers will be testing it for accuracy and whether it can help them make precise measurements of how corals change, grow and respond to disturbances.

AIMS senior research scientist Dr Renata Ferrari said the recent mass coral bleaching event on the Reef, the fifth since 2016, made it more important than ever to explore tech innovations like Reef-NeRF and Reef-3DGS.

“With disturbances on the Reef occurring more frequently and intensely because of climate change, we are in a race against time to understand how coral communities are changing, growing and responding,” said Dr Ferrari.

“Digital twins produced via photogrammetry have been helping us to do this at AIMS, but this is a technology that is decades old. If Reef NeRF helps us to understand the Reef more precisely and efficiently, this information can help guide approaches that support coral reef recovery and adaptation to warming oceans, and will help to improve the success of interventions.”

A new 3D digital twin of a section of the reef was created by Professor Wei Xiang, Director of the Cisco-La Trobe Centre for AI and Internet of Things (IoT) and Postdoctoral Research Fellow Dr Kang Han.

Professor Xiang said that by using multiple 2D images, Reef-NeRF/3DGS will aim to push the boundaries of 3D underwater reef modelling and provide unprecedented accuracy.

“This AI-empowered software is tailored to the unique challenges of reconstructing high-quality models of the complex underwater environments of coral reefs,” he said.

“By leveraging La Trobe’s home-grown, cutting-edge AI technology, we aim to answer questions around how climate change may impact the structure of coral reefs. This will pave the way for more effective restoration strategies.”

Professor Xiang added the project held promise for enhancing public awareness and engagement in marine science, empowering them “to better understand the importance of coral reefs and the urgent need for their protection.”

AIMS’ lead data scientist Mathew Wyatt said the collaboration would enable AIMS to stay up to date with the technological progress in AI-enabled modelling.

“NeRF and 3DGS are hot technologies at the moment, and La Trobe are Australian leaders in this area of research. AIMS wants to remain at the forefront of technological innovation when it comes to reef monitoring, so it’s a no brainer for our scientists to be engaging in this technology and partnership at this point in time,” said Mr Wyatt.

“By harnessing the power of AI, high performance computing and interdisciplinary research, we are aiming to unlock new insights into coral reef ecosystems. This will help safeguard these natural treasures for future generations.”

Cisco ANZ Director for Education and Strategic Industries Reg Johnson said: “Cisco is pleased to be supporting the research of La Trobe and AIMS to evaluate the impact of climate change on the reef and its recovery. The Cisco Centre for AI/Internet of Things at La Trobe is playing a critical role in enabling the use of emerging technology to solve environmental challenges.”