Autonomous Satellite-based Surveys of Antarctic Seals Using Multi-scale Convolutional Neural Networks

As a group, pack ice seals are the largest consumer of Antarctic krill in the Southern Ocean and provide a lens through which we can understand how the Southern Ocean food web is changing in response to climate change and the krill fishery. Unlike penguins, which breed in fixed colonies and are easily found year after year, pack ice seals haul out unpredictably on the pack ice, making comprehensive surveys by ship or even fixed wing aircraft nearly impossible. The proliferation of high resolution commercial satellite imagery has generated a scientific boom for Antarctic biologists and satellite-based surveys of penguins are increasingly being used in areas where direct surveys are difficult. Being spread over much larger areas, the survey of pack ice seals will require a new approach that combines tools from remote sensing (e.g., object-based image analysis), visual computing, and deep learning. The imagery to systematically survey the entire Antarctic coastline is available. What is required at this step are new tools that harness deep learning in the context of satellite image interpretation. At a course level, we flag groups of seals using Squeezenet, a convolutional neural network (CNN) model previously trained for general object recognition. In contrast with most current CNN architectures, Squeezenet is very compact and uses relatively few parameters, giving it a faster running time at the cost of accuracy – a necessary sacrifice given the scale of our survey. With seal haul-out locations in hand, we can sweep through our imagery again, at finer scales, now targeting individual seals inside seal haul-outs with a simple blob detector. Scaling this procedure up to provide regular continental-scale monitoring will require not only advances in computer vision but also efficient parallelization across high performance computing resources. Continental scale surveys of pack ice seals would represent a major advance for Antarctic biology, and would provide critically needed information on the abundance of a key krill predator. Moreover, the computational tools required to automate object detection in sub-meter resolution satellite imagery will accelerate the adoption of computer vision techniques in remote sensing.