Machine Learning for Automated Detection of Shipwreck Sites from Large Area Robotic Surveys
May 23 - June 3, 2022 / June 5 - 23, 2023
Expedition Summary
Due to its maritime history and strategic location, Thunder Bay National Marine Sanctuary contains almost 100 known shipwreck sites, with over 100 shipwrecks still left to be found. Shipwrecks can help us better understand our past, but discovering and exploring them is expensive, time-consuming, and labor intensive. By advancing and training the capabilities of marine robotic systems to search for and survey shipwreck sites autonomously, scientists aim to increase the efficiency and decrease the costs associated with such exploration efforts. In 2022 and 2023, a multi-institute team of researchers conducted expeditions in Thunder Bay to do just that.
Part 1: May 23 - June 3, 2022
The main goal of the first year of field expeditions was to collect datasets to develop machine learning methods for shipwreck detection from sonar imagery. The team also conducted preliminary imaging surveys for close-range inspection of shipwreck sites with robotic systems.
Field expeditions were focused in Thunder Bay National Marine Sanctuary (TBNMS), which is unique in its abundance of known shipwreck sites of varying type, size, and wreck conditions. The team conducted eight shore-based day operations throughout the sanctuary on R/V Storm between May 23-June 3, 2022. Each mission consisted of traveling to the initial starting point, deploying robotic systems, and conducting surveys in a local area. When the mission was complete, we moved to the next planned site.
To collect side-scan sonar data, we deployed Michigan Technological University’s IVER-3 autonomous underwater vehicle (AUV). Surveys covered a total of 8.04 square kilometers (3.1 square miles) over the course of field trials. Figure A shows an example of the high-resolution side-scan sonar imagery we collected with the AUV over the Grecian site.
Working with maritime archaeologists at the sanctuary, we created high-resolution labels of shipwreck sites, distinguishing between background, ships, debris, and moorings. Figure B shows a sample label. Labeled data allows us to train and evaluate machine learning methods for detecting shipwreck sites in side-scan sonar imagery.
We also conducted preliminary deployments of a remotely operated vehicle (ROV) for close-range imaging surveys of shipwreck sites. Over the course of the expedition, we had a total of 13 ROV deployments with three different ROV platforms. With data collected by the ROV, we are able to create large-scale 3D reconstructions of shipwreck sites. Figure C shows initial 3D reconstructions obtained from data collected by the ROV.
The data collected during the Year 1 field expeditions will lay the foundation for developing and training machine learning methods in Year 2 of the project.
By Katie Skinner, Assistant Professor, University of Michigan
Part 2: June 5 - 23, 2023
Building on the previous year’s work, the objectives of the second year of this expedition were to validate and demonstrate our shipwreck detection and exploration pipeline, and to continue to collect sidescan data of shipwreck sites in Lake Huron. In addition, a secondary mission was to perform exploratory searches for shipwrecks in areas with high potential for new discovery.
Field work took place in the Thunder Bay National Marine Sanctuary. Figure 1 shows an overview of the planned survey regions, though modifications were made to these survey regions to accommodate weather and vehicle operating status each day.
Operations consisted of shore-based day expeditions between June 5-9, June 13-16, and June 20-21, 2023, during which we visited 15 sites. Each mission consisted of traveling to the initial waypoint, deploying robotic systems, and conducting surveys in a local area. When the mission was complete, we moved to the next planned waypoint. For remotely operated vehicle operations, we used moorings to ensure safe operation.
As a collaboration between the University of Michigan, Louisiana State University, and Michigan Technological University, the full detection pipeline involved an initial survey from an IVER3 autonomous underwater vehicle (AUV) equipped with a side-scan sonar followed by our novel shipwreck detection model which produced GPS coordinates of potential wreck sites (see Figure 2).
These coordinates were passed along to an autonomous planning and navigation algorithm on a BlueROV, which was used to collect up-close camera and depth data for 3D reconstruction. Figure 3 shows the survey path that was automatically generated by algorithms developed by the Louisiana State University team.
Once the survey was planned, the modified BlueROV, Dory, was deployed to conduct an autonomous survey of the site. Figure 4 shows deployment of the Dory AUV from the R/V Storm research vessel, and Figure 5 shows imagery collected during an autonomous survey of the Barge No. 1 shipwreck site.
Overall, this field expedition successfully demonstrated the full pipeline developed by the multi-institute team to deploy robotic platforms to automatically detect shipwreck sites from sidescan sonar imagery, and then perform planning and navigation to carry out autonomous surveys of detected shipwreck sites. The collected data will be used to compute 3D models of surveyed sites.
By Anja Sheppard (Ph.D. Student, University of Michigan) and Katie Skinner (Assistant Professor, University of Michigan)
Published February 8, 2024
Related Links
Education Themes
Media Contacts
Libby Haydel
Louisiana State University Media
ehaydel1@lsu.edu
Marcin Szczepanski
University of Michigan Media
marcins@umich.edu
Gabe Cherry
University of Michigan Media
gcherry@umich.edu
Cyndi Perkins
Michigan Technological University Media
cmperkin@mtu.edu
Stefanie Sidortsova
Michigan Technological University Media
ssidorts@mtu.edu
Emily Crum
Communication Specialist
NOAA Ocean Exploration
emily.crum@noaa.gov
Funding for this expedition was provided by NOAA Ocean Exploration via its Ocean Exploration Fiscal Year 2021 and 2023 Funding Opportunity and via the University of Michigan Robotics Institute Fellowship program.