Optical Imagery is powerful for detecting vessels of many sizes, including those that are quite small, particularly when a wake or bottom disturbance is visible. The technical term for Optical Imagery is Electro-Optical Imagery. You may also see/hear the abbreviation, EO. 

How it works: Optical Imagery generates a color image, similar to a personal camera (e.g., phone camera). The Optical imagery coming from satellites, such as Sentinel-2, offer multi-spectral instrument with multiple spectral bands, or individual color channels. Skylight uses natural color images (red=red, green=green, and blue=blue). 

Other combinations called “false color” can assign other observed bands, such as infrared, to visible color channels in the image. These other bands are visible in Sentinel-Hub.

Value / Challenges: Optical images display in color, including the surrounding context (land, wake, etc.). The additional context, especially a vessel wake, can help to identify smaller objects/vessels.

Clouds pose a challenge in two ways: blocking visibility completely or small clouds looking similar to vessels. Small islands, whitecaps, icebergs and other issues can cause false positives.

Satellites and resolution

Skylight currently processes two sources of regularly available Optical Imagery.

The first is Sentinel-2 from the European Space Agency's Copernicus program. The second is two similar satellites, Landsat 8 and Landsat 9 from the US  National Aeronautics and Space (NASA). The imagery from these satellites is similar in resolution, coverage and revisit rate. 

The full images from Sentinel-2 and Landsat 8/9 data are available on the Sentinel-hub browser as well as the Copernicus browser. You can track the paths of these satellites on Spectator Earth.

Notably, these satellites provide greater coverage to more small island EEZs compared to Sentinel-1 (Satellite Radar). 

Source: Sentinel-2

Sentinel-2 has two satellites (Sentinel-2A and Sentinel-2B). Another satellite (Sentinel-2C) was launched.

Key stats

• Coverage: At least partial coverage of most EEZs.
• Resolution: 10 meters 
• Latency: 3-6 hours on average
• Revisit rate: 5 days

Coverage

Sentinel-2 satellites collects data from many continental and island EEZs, as well as som areas outside of EEZs (e.g., Mariana Trench). See image below of the frames indicating the total coverage available and processed by Skylight (by both Sentinel-2 and Landsat 8,9. More on Landsat below). These are all the relevant frames to the maritime space (i.e. non-terrestrial).

Imaging occurs late morning/early afternoon local time due to the sun-synchronous orbit of the satellites.

Resolution

The resolution of Sentinel-2 imagery is 10 meters. Each pixel in the associated image is 10m x 10m.

The image chip created for each vessel detection is 1280m x 1280m (less than 1% of the total image frame). This reference can help estimate the approximate size of the vessel. 

Images may appear slightly hazy. This is due to Skylight using the “top of atmosphere” image (Level 1C product) to minimize the time from imaging to making vessel detections available in the platform. 

Latency

The average latency (delay from time of imaging) for Optical Imagery from Sentinel-2 is 3-6 hours. 

This range includes the time for the satellite to capture the image, send the image to earth, and then for Skylight to process the data. The time to send the image from the satellite to earth accounts for a majority of the latency.

The chart below is a 2 week sample. Note that the day-to-day average latency is mostly between 3-6 hours, but sometimes as short as 2 hours and sometimes more than 7 hours.

Revisit Rate

The revisit rate of Sentinel-2 imagery is 5 days thanks to a combination of two Sentinel-2 satellites working in tandem. 

The satellite's path crosses in some locations (e.g., part of the Caribbean, Mediterranean) that result in more frequently available data, though these are coming from different swaths/paths.

Landsat 8,9

Landsat 8 and Landsat 9 are two individual satellites with a similar optical imagery sensor. Landsat 7 is no longer imaging the earth, as of May 2024.

Key stats

• Coverage: At least partial coverage of most EEZs.
• Resolution: 15 meter black and white, 30 meter color
• Latency: 3-7 hours on average
• Revisit rate: 8 days 

Coverage

Landsat 8,9 satellites collects data from many continental and island EEZs. See above image in the Sentinel-2 section for visualization of global coverage.

Imaging occurs late morning/early afternoon.

Resolution

Landsat sensors include a 15 meter black and white band and 30 meter color band. The chip images shown by Skylight is color applied to black and white image in a process called pan-sharpening. 

Latency

The average latency (delay from time of imaging) for Optical Imagery from Landsat 8,9 is 3-7 hours. 

This range includes the time for the satellite to capture the image, send the image to earth, and then for Skylight to process the data. The time to send the image from the satellite to earth accounts for a majority of the latency.

The chart below is a 2 week sample. Note the range for average day to day latency is mostly between 3-7 hours, but sometimes as short as 2 hours and sometimes more than 7 hours.

Model information

The models used for Sentinel-2 and Landsat 8,9 imagery are different. Both models are continuously evaluated and revised based on feedback and additional training data. Below are some details.

Sentinel-2

The Sentinel-2 vessel detection model uses a Faster-RCNN detection head coupled with much larger Swin Transformer that has sufficient parameters to perform complex analysis of the S2 bands.

We found that pre-training the backbone on SatlasPretrain, a large-scale remote sensing dataset, further improved performance.

The S2 model has an F1 score of 0.81.

Training data: maritime experts annotated 43,102 vessels (point labels) in S2 imagery.
The various S2 bands provide rich information about the physical objects present in a scene. For example, RGB bands already enable distinguishing most vessels from other marine objects, but additional bands can be leveraged to further improve accuracy due to the different reflectance signatures of vessels and other objects. Thus, we developed a detection model that, like our S1 model, 

Landsat 8,9

The vessel detection computer vision model for Landsat 8,9 imagery uses a two-stage detection model. The first stage employs a detector with a Swin-v2-Base encoder and Faster R-CNN head to identify potential vessel locations using bands B2 to B8 (covering RGB bands and shortwave infrared wavelengths). To enhance accuracy of the detections, the second stage uses a Swin-v2-Base classifier that examines the patches around the detected vessels to confirm their presence.

Together, the two-stage pipeline significantly improves detection accuracy compared to using the detector alone. Besides the two-stage model, we also applied filters like marine infrastructure and cloud/ice masks to exclude known false positives.

For Landsat, the detection model achieved an F1 score of 76.0% and the classification model achieved an F1 score of 72.2%.

Training data: The first stage of the model is trained on 18,509 vessel labels from 7,954 patches, and it uses overlapping sliding windows and non-maximum suppression to manage large scenes and reduce duplicate detections. 

The second stage classifier is trained on approximately 2,000 expert-annotated detections.