Using digital twins to advance the management and restoration of the Great Barrier Reef

Joshua Parker and Renata Ferrari, EcoRRAP

Imagine virtually exploring parts of the Great Barrier Reef in 3D without ever getting wet.

Photogrammetry is the scientific technique that can not only transport you into the delicate corners and crannies of the underwater world but enable AIMS scientists to measure large sections of the Great Barrier Reef. This is accelerating their ability to help vulnerable reefs recover and adapt to a warming climate.

An interactive digital twin from Davies Reef, central Great Barrier Reef, taken by the EcoRRAP team.

What is photogrammetry?

Photogrammetry is the science of measuring objects from images.

Photo = image

grammetry = precise measurement

It is used across many disciplines and ecosystems to produce intricate image reconstructions (i.e., digital twins) of important things.

The process uses photographic sensors mounted onto a wide range of platforms, such as satellites, drones, boats, or hand-held devices, to produce two- and three-dimensional terrain digital twins.

Photogrammetry was first developed in 1851 for terrestrial topography. Underwater cameras were widely adopted into research in the early 2000’s, however modern photogrammetry has seen impressive growth in marine ecosystem monitoring and research in the last 10 years. It is quickly becoming a standard for marine ecosystem monitoring because it improves the speed, precision and accuracy of data used for management and conservation.

Digital twins are an incredible tool that allow us to understand the world, how and why it is changing, in a better and more efficient way. One of the many innovative ways photogrammetry is being used is by cetacean researchers, who are producing digital twins of whales to assess their size, body condition and health in a non-invasive way.

Dive, click, crunch the data - creating digital twins of coral reefs

AIMS has been using digital twins across multiple programs since 2017 to ‘diagnose’ the health status of reefs and to guide approaches that support their recovery and adaptation to a warming ocean.

To generate digital twins of coral reefs, researchers need to first image the reef. This is done with underwater cameras in waterproof housings as seen above. In some cases, science teams tow several cameras from boats or use underwater robots to capture these images. The images that create the most detailed digital twins are achieved by SCUBA divers who can get within centimetres of the reef and hone in on specific patches of the reef. To ensure precision in the measurements, markers are also placed on the reef and photographed. Markers are objects of known sizes and orientation, helping bring the images together during processing and providing an important scale reference.

Once several thousand images have been successfully captured, they are quality checked and backed up on board an AIMS research vessel. Back on dry land, the images are processed using a super-computer which typically takes six hours to produce a 72 square metre digital twin of a patch of reef, just like the one made on Davies Reef (above), located 100 kms north-east of Townsville.

Understanding small but important changes on reefs using digital twins

Coral reefs are incredibly complex, intricate structures. While scientists have been investigating them for many decades, it has been difficult to measure the finer details of how corals change, grow and respond to disturbances. This information is crucial to understanding the potential of coral reef communities to recover from climate change impacts. Traditional methods to do so are quite labour intensive, so only a few colonies or small sections of reef can be monitored.

Photogrammetry has changed things. Scientists are now using the technique to capture the changes in coral communities at many reefs across the Great Barrier Reef and into the Torres Strait. The Ecological Intelligence for Reef Restoration and Adaptation Program (EcoRRAP) implemented the use of photogrammetry in 2021 to measure fine scale changes within coral reef communities and understand the dynamics of different coral types across different reef habitats and environmental conditions. This dataset also includes a mass bleaching event in 2022, allowing the EcoRRAP researchers to understand how the severity and extent of bleaching varies across coral type, coral size and environment.

This information will help scientists from the Reef Restoration and Adaptation Program work smarter, not harder when it comes to future reef restoration approaches. If we know where corals are most vulnerable to bleaching and in which environments they perform best, scientists can outplant corals on reefs where they will bleach least, grow fastest, and reproduce most. This will help increase the success of restoration interventions.

This is just like understanding the impact of COVID-19 on different groups of people and where they work. Scientists and health officials were able to use the information to prioritise vaccinations for those most vulnerable (eg the elderly) and those most exposed (eg health workers).

How else are we using photogrammetry?

Across AIMS, more than 6,000 hours and 1,000 digital twins have been produced to date, giving our teams a greater ability to investigate coral reefs than ever before.

AIMS’ Long-Term Monitoring Program (LTMP) was the first program to use modern photogrammetry to record the structural complexity of reefs at different times, to gain greater insights into reef health and potential trajectories following disturbances. The data generated from this project is improving our ability to predict the impacts of environmental disturbances on coral reefs.

Other AIMS programs including, ReefScan and ReefSong teams have used modern photogrammetry since 2022. The 2.5D models created by autonomous vehicles and optical imaging systems mounted on AIMS research vessels for ReefScan are suitable for broadscale monitoring of entire coral reefs, crown-of-thorns starfish hotspots and seagrass meadows.

The ReefSong project, part of the Australian Coral Reef Resilience Initiative, is building high-resolution 3D models of experimental patches on the Ningaloo Reef and Lizard Island. This is helping investigations on how fish influence coral settlement and individual coral growth over time.

Another application of modern photogrammetry for restoration research is to track the movement of several different shapes and designs of coral seeding devices over time to improve device retention on the reef.

The future - integrating AI for identification

Photogrammetry is being paired with Artificial Intelligence (AI) to accelerate the rate of extracting information from digital twins and thus provide timely information to management and conservation.

AI is being developed at AIMS which will increase the speed at which we can translate digital twins into knowledge to inform reef management and conservation. For instance, AI can already provide greater efficiency in the repetitive task of tracing individual coral colonies and the classification of seafloor categories.

Reefcloud offers a platform for large amounts of underwater images from around the world to be uploaded and analysed with great accuracy and speed. It is regularly being used in the Pacific Islands to facilitate marine management decisions as well as at AIMS to process large volumes of images of the Reef at high resolution taken by LTMP, EcoRRAP and ReefScan platforms.

Both the LTMP and EcoRRAP are collaborating with the TagLab team to significantly reduce the human effort required to manually trace individual coral colonies. This gives the teams the ability to ‘train’ the program to classify and trace objects of interest with great accuracy, significantly reducing the processing time and data extraction of each digital twin.

More data for better understanding and effective solutions

Photogrammetry provides more detailed information faster on how reefs change over time, which gives better-informed and more timely decisions.

Digital twins allow us to measure new things, unattainable through traditional methods. Large real-world problems require multidisciplinary solutions and digital twins facilitate data standardisation, sharing information and collaboration, which in turn will lead to better solutions to assist reef recovery or at least delay their collapse increasing the chances for adaptation at scale.

Digital twins can also be used for better communication, providing an interactive platform to tell stories about a particular reef with Traditional Owners, decision makers, future generations and the wider public.