Unmanned aerial vehicles and deep learning for assessment of anthropogenic marine debris on beaches on an island in a semi-enclosed sea in Japan (2024)

The rapid increase in anthropogenic marine debris has been recognized as a global environmental problem (Ostle et al 2019). The majority of marine debris consists of plastic (Barnes et al 2009), and it is estimated that 4.4–12.7 million metric tons of plastic are discharged into the ocean annually (Jambeck et al 2015). Previous studies have indicated that in the western Pacific Ocean, particularly in the Kuroshio Current area, more than 3.5 million pieces of debris per square kilometer at the ocean surface may be present (Yamash*ta and Tanimura 2007). The problem with marine debris is not only the amount of inflow but also its persistence. Plastics are chemically resistant to degradation and remain in the marine environment for hundreds to thousands of years (Derraik 2002). Therefore, they continue to accumulate over a long time and have a variety of effects, such as an impact on tourism due to aesthetic degradation and ingestion by marine organisms. In addition, plastic fragmentation and degradation driven by environmental factors such as ultraviolet (UV) light result in small particles that are known as microplastics (Andrady 2011, Cole et al 2011).

Marine debris affects the human economy, particularly, shipping, fishing, and tourism (McIlgorm et al 2011). For example, in tourist destinations, aesthetic degradation by marine litter decreases revenue from tourism, and local authorities need to bear the cost of cleanup (Iñiguez et al 2016). The dynamics of marine debris widely vary, and heavier items may accumulate on the seafloor. There have been attempts to collect marine debris, but these are costly and sometimes dangerous. For instance, in a project to remove debris from the seabed in South Korea, about 460 tons of trash were recovered at a cost of $3.8 million, and fatal incidents were involved (Cho 2011).

Marine ecosystems are also affected by marine debris. Ingestion and entanglement are two impacts of marine debris on marine organisms (Derraik 2002). Marine organisms from a wide range of taxonomic groups ingest marine debris, sometimes leading to death (Gregory 2009). Entanglement in discarded fishing gear-related debris is known as 'ghost fishing' and is recognized as a serious threat. At least 690 species are threatened by marine debris, of which at least 17% are classified as near-threatened or higher on the IUCN Red List (Gall and Thompson 2015). To mitigate the impact of marine debris, a quantitative assessment of the distribution of marine debris is the first step. Once the location and amount of marine debris present are understood, the removal of marine debris can be implemented.

The accumulation of marine debris has been confirmed over a wide area from the Arctic to the Antarctic and in the deep sea (Woodall et al 2014). However, the amount of plastic debris observed in the ocean is much lower than that previously expected (Cózar et al 2014). Various studies have been conducted to determine where the 'missing plastic' exists (Law and Thompson 2014). This 'missing plastic' may be drifting in currents (Eriksen et al 2014) or water columns (Dai et al 2018) or is accumulating on the seafloor (Chiba et al 2018). However, the importance of focusing on beaches has been highlighted (Brennan et al 2018, Olivelli et al 2020). About half of the world's population lives near the coast, making coastal areas hotspots for microplastic pollution (Cole et al 2011). Shorelines are also strongly influenced by terrestrial factors such as stormwater and road distribution (Willis et al 2017), and backshore areas with vegetation can be a sink for large marine debris (Olivelli et al 2020). In addition, plastic products on beaches are exposed to UV light that causes photo-oxidative degradation (Singh and Sharma 2008 Sathish et al 2019). On beaches in the daylight, the thermal degradation of plastic progress faster than in seawater (Andrady 2011).

In recent years, unmanned aerial vehicles (UAVs) have been used in a wide range of research fields (Jiménez López and Mulero-Pázmány 2019). They are also used to survey marine debris on beaches (Kako et al 2012, Bao et al 2018, Deidun et al 2018, Andriolo et al 2020, Gonçalves et al 2020, Lo et al 2020, Merlino et al 2020). UAVs have the advantage of cost effectiveness and the ability to obtain high-resolution images. Studies have conducted marine debris surveys using satellite images, but the high cost and low resolution of the data have been issues. For example, WorldView-4 satellite imagery, with a resolution of about 0.3 m, cannot detect the majority of marine debris on beaches (Topouzelis et al 2019). UAVs also facilitate surveys in difficult to access areas such as islands (Lavers and Bond 2017). In addition, UAVs can be used to collect substantial data using a standardized method and compare the distribution of marine debris from different locations. However, the analysis of image data taken by UAVs remains labor-intensive (Jiménez López and Mulero-Pázmány 2019). Deep learning can improve the efficiency of data processing and analysis. However, research on the automatic identification of marine debris from UAV images is a new field, and there are few previous studies on this topic (Martin et al 2018, Fallati et al 2019, Kako et al 2020, Martin et al 2021, Papakonstantinou et al 2021).

In this study, we aimed to identify anthropogenic marine debris on a beach via high-resolution UAV images using deep learning. The challenges we hope to address in this study are (1) whether UAV images are suitable for automatic identification by deep learning and (2) whether identifying anthropogenic marine debris from the variety of objects present on the beach is possible. If these can be accomplished, it will be possible to survey a wide area with a standardized protocol, which could significantly improve the world's future marine debris management.

Study area

UAV survey protocols

The deep learning algorithm

The deep learning model successfully identified marine debris in the high-resolution images (figure 3). For data from Narugashima Island in December 4, the sensitivity was 59% and the precision was 90% (table 2). On the other hand, for the data taken in January 25, the sensitivity was 43% and the precision was 62%. The average of both results was 51% for sensitivity and 76% for precision. The main FPs were waves and rocks (figure 4), and when the marine debris contacted driftwood, it was not detected (figure 5).

As a result of the identification by our deep learning model, the numbers of marine debris at the site on December 4 were 53, 297, and 118 pieces in the north, central, and south parts of the beach, respectively (table 3). The mean number of marine debris (±standard error (SE)) was 156 pieces (±72.95 pieces). On January 25, the numbers of marine debris at the comparison site were 80, 30, and 93 pieces in the north, central, and south parts of the beach, respectively. The mean number of marine debris (±SE) was 68 pieces (±19.20 pieces). On December 4, the marine debris density was 0.12, 0.66, and 0.26 pieces/m² in the north, center, and south, respectively. The average marine debris density (± SE) was 0.35 pieces/m² (±0.16 pieces/m²), and the marine debris density on January 25 was 0.18, 0.06, and 0.21 pieces/m² in the north, center, and south, respectively. On this observation date, the average marine debris density on this observation date (±SE) was 0.15 pieces/m² (±0.04 pieces/m²).

The areas of marine debris decreased more on January 25 than on December 4. By calculating the actual length of the pixels, the area of the debris could be calculated from the images. The actual length of one pixel (±SE) was 1.06 mm (±0.04 mm). Based on this length, the areas of the blue frames on December 4 were 2.77, 12.81, and 3.94 m² in the north, central, and south part of the beach, respectively. The average blue-framed area (± SE) was 6.51 m² (±3.17 m²). On January 25, the areas of the blue frames were 3.95, 4.26, and 6.12 m² in the north, central, and south beach, respectively. The average blue-framed area (± SE) was 4.78 m² (±0.68 m²). The mean marine debris percentage in the blue-framed area (± SE) was 62.6% (±1.77%). Based on this percentage, the areas of marine debris on December 4 were 1.74, 8.02, and 2.47 m² in the north, center, and south, respectively. The average area (±SE) was 4.08 m² (±1.99 m²). The areas of marine debris on January 25 were 2.48, 2.67, and 3.83 m² in the north, center, and south, respectively. The average area (± SE) was 2.99 m² (±0.42 m²).

The area coverages could also be estimated from the images. The area coverages of marine debris were 0.41%, 1.79%, and 0.54% in the north, central, and south parts of the beach in December 4. The average area (± SE) was 0.92% (±21.16%). Similarly, the area of marine debris was 0.57%, 0.56%, and 0.86% at the three parts of the beach, respectively, in January 25. The average area (± SE) was 0.66% (±3.88%). The number and area coverages of marine debris were lower in January compared to December. In addition, it was demonstrated that deep learning could identify marine debris from UAV images and that changes in the debris could be quantified.

In this study, high-resolution images of marine debris were acquired by combining a UAV equipped with a zoom lens and low altitude flight operation. In addition, the study showed that UAV images can be used for the object detection of marine debris. Although UAVs have also been used in marine debris surveys, high-resolution image acquisition was a challenge. For example, the resolution in the previous study was 4.4 mm/pixel when the DJI Phantom 4 was used at 10 m and 5.0 mm/pixel when it was used at 17 m (Fallati et al 2019; Kako et al 2020). In this study, a resolution of 1.1 mm/pixel was obtained using the DJI Mavic 2 zoom from an altitude of 5 m. The resolution of the present study was high compared to previous studies, and it was achieved using the combined approach of a UAV that was equipped with the zoom lens and operated from a low altitude.

The advantage of UAVs is that the vehicles can perform survey at a low cost, so periodic monitoring can reveal the spatio-temporal distribution of marine debris. In this study, we acquired 4,421 images and performed an intensive survey at a low altitude to obtain high-resolution images from the target area. Our survey over an area of 0.81-ha was implemented by replacing the battery 12 times. To the best of our knowledge, this is the first study in which a UAV equipped with a zoom camera was operated at a 5 m altitude to identify marine debris.

Our model achieves a sensitivity of 0.59 because of the high-resolution images from the UAV. Moreover, our model could identify anthropogenic marine debris from the wide variety of objects on a beach. Training data containing multiple types of debris enabled the high performance of our model. In addition, we annotated two classes, anthropogenic marine debris and natural objects, allowing the model to distinguish between human and natural-origin debris. The types and composition of these categories differ, and annotating typical natural objects improved the performance of the model.

Our technique evinced a decrease in marine debris through cleanup activities. These activities have been performed worldwide on beaches. However, continuously monitoring a large area after cleanup activities is labor-intensive. In our study, we found that combining UAVs and deep learning makes it possible to compare the amount of debris before and after a cleanup activity and quantify the effectiveness of the cleanup effort. Moreover, we clarified that the distribution of marine debris varied depending on the site. Ocean currents and wind direction may affect marine debris distribution. In the future, continuous surveys using this technique might reveal the factors behind marine debris accumulation.

Despite using the same model, the identification accuracy differed, as sensitivity decreased when the test images were taken on January 25. The reduction in the amount of marine debris on the beach may affect the model's sensitivity, and more marine debris could be detected by changing the threshold of our model. In addition, detecting entangled debris in vegetation was challenging. Furthermore, the orientation of the marine debris may affect the results. Debris entangled in vegetation can be scattered at various angles, and the training images did not contain these varied angles. Therefore, improving the training images will improve the model performance.

Our model underestimated the amount of marine debris owing to a high false negatives. Previous studies have shown that identifying marine debris using UAVs also underestimates the amount of debris compared to a human ground survey (Martin et al 2018). The shadows of debris can also affect identification, and it is recommended that UAVs be operated around noon to reduce shadows (Martin et al 2018; Lo et al 2020). In our study, the images were consecutively taken between 10:00 and 15:00. Therefore, the high false negatives may be attributed to the Sunlight conditions in each image.

Some limitations still exist in this study. First, detecting small anthropogenic marine debris such as fragments of plastic products is difficult. The criteria for the annotation also affect the low detection of small debris. Marine debris was annotated if the size was larger than 100 pixels because it was difficult to annotate all of the debris. Further consideration of criteria for the annotation will be needed to detect the smaller size of debris. Second, the results of our study show lower sensitivity, indicating that the model cannot detect all marine debris because of the excesses of false negatives. A variety of anthropogenic marine debris was annotated as one class, such as PET bottles, plastic bags, and cans. Therefore, separating the class into more subdivided categories may improve the sensitivity. Although high-resolution images acquired operated UAV at a low altitude, this operation required multiple batteries; for example, batteries were replaced 12 times in this study. In addition, operations over several hours also affect the light conditions of the image; thus, this difference may have affected the performance. Additionally, object detection was used in this study, but it may be necessary to consider other approaches, such as semantic segmentation for more accurate estimation. Finally, our results show microplastic detection is quite challenging; however, monitoring these distributions is essential to understanding plastic pollution in the marine environment. Citizen science can also be beneficial to survey microplastic pollution with other protocols.

Although beaches attract attention as places that often accumulate marine debris, few studies have identified marine debris on beaches using UAVs combined with deep learning (Fallati et al 2019). Some studies used aircraft to survey the distribution of marine debris (Moy et al 2018); however, the cost is prohibitive. The advantage of UAVs is that they can acquire higher-resolution images at a lower cost than aircraft. In addition, UAVs enable safe surveys in dangerous areas where marine debris accumulates, such as areas containing medical waste. In these areas, remote sensing surveys of beaches using UAVs can be conducted. Using UAVs is also beneficial from an ecological perspective (Harris et al 2015). Beaches are essential for the breeding of some species of marine wildlife. For example, the human use of beaches affects areas where sea turtles lay their eggs (Antworth et al 2006) and the behavior of seabirds (Burger and Niles, 2013). Thus, minimizing disturbance to marine wildlife by conducting surveys without directly entering the beach is possible.

The present study, showed that deep learning could identify marine debris obtained from high-resolution UAVs images. Marine debris is expected to increase in the future, and the monitoring is an urgent global issue. Using UAVs and deep learning will allow for uniform surveys to be conducted worldwide at a low cost. To help solve the marine debris problem, it is necessary to clarify the current situation at a global scale. Citizen science in obtaining marine debris has continued to grow (Sandahl and Tøttrup 2020), and our model can be used to evaluate cleanup activities. The worldwide use of methods such as those described in this study, which combines field surveys and information science technology, will help enable the efficient management of marine debris.