The search for dark vessels: AIS / ship detection fusion workflow¶

All ships above a certain size are by law required to use an AIS (Automated Identification System) transponder and are therefore trackable at all times wherever they go. A ship without such a signal is called a "dark vessel". While there are not always sinister reasons for this behaviour this quite often indicates that something "fishy" is going on. Not surprisingly there are many organisations which are interested in this kind of information.

In the following we show how dark vessels can be identified by combining a Machine Learning-based algorithm working on satellite images (provided by Airbus, see ship detection block on UP42 marketplace for more details) with AIS signals (provided by ExactEarth on UP42).

The workflow consists of the following steps:

Get SPOT imagery for the given AOI
Execute format conversion, CRS conversion, tiling, ship detection and ship identification blocks via parallel jobs
Visualize the results

To run the example costs around 8,000 UP42 credits. Disclaimer In a real-world scenario we would only use SPOT Streaming data as input for the shop detection and identification. In the context of this analysis we are using SPOT Download data so we can download it (which the Streaming data license does not allow) and produce attractive visualisations. Using streming data would result in a much lower number of credits.

Setup¶

Import required libraries

In [258]:

            
                Copied!
                
                    
                    
                
                

        
import up42
import geopandas as gpd
import rasterio
from rasterio.plot import show
import matplotlib.pyplot as plt
from shapely.geometry import box
import geojson
import up42
import geopandas as gpd
import rasterio
from rasterio.plot import show
import matplotlib.pyplot as plt
from shapely.geometry import box
import geojson

Configure areas of interest

In [259]:

            
                Copied!
                
                    
                    
                
                

        
aoi_sby = {"type":"FeatureCollection","features":[{"type":"Feature","properties":{},
                "geometry":{"type":"Polygon","coordinates":[[[112.713691,-7.183133],
                                                             [112.73185,-7.183133],
                                                             [112.73185,-7.172643],
                                                             [112.713691,-7.172643],
                                                             [112.713691,-7.183133]]]}}]}
aoi_gib = {"type":"FeatureCollection","features":[{"type":"Feature","properties":{},
                "geometry":{"type":"Polygon","coordinates":[[[-5.370973,36.115663],
                                                             [-5.348018,36.115663],
                                                             [-5.355756,36.141326],
                                                             [-5.3598,36.147976],
                                                             [-5.370531,36.148606],
                                                             [-5.370973,36.115663]]]}}]}
aois = [{'title': 'SBY', 'geometry': aoi_sby},
        {'title': 'GIB', 'geometry': aoi_gib}]
aoi_sby = {"type":"FeatureCollection","features":[{"type":"Feature","properties":{},
                "geometry":{"type":"Polygon","coordinates":[[[112.713691,-7.183133],
                                                             [112.73185,-7.183133],
                                                             [112.73185,-7.172643],
                                                             [112.713691,-7.172643],
                                                             [112.713691,-7.183133]]]}}]}
aoi_gib = {"type":"FeatureCollection","features":[{"type":"Feature","properties":{},
                "geometry":{"type":"Polygon","coordinates":[[[-5.370973,36.115663],
                                                             [-5.348018,36.115663],
                                                             [-5.355756,36.141326],
                                                             [-5.3598,36.147976],
                                                             [-5.370531,36.148606],
                                                             [-5.370973,36.115663]]]}}]}
aois = [{'title': 'SBY', 'geometry': aoi_sby},
        {'title': 'GIB', 'geometry': aoi_gib}]

Authenticate with UP42

In [260]:

            
                Copied!
                
#up42.authenticate(project_id="123", project_api_key="456")
up42.authenticate(cfg_file="config.json")
up42.settings(log=False)
#up42.authenticate(project_id="123", project_api_key="456")
up42.authenticate(cfg_file="config.json")
up42.settings(log=False)

Catalog Search¶

Search cloudfree SPOT images for the two aois and visualise the quicklooks.

In [261]:

            
                Copied!
                
                    
                    
                
                

        
catalog = up42.initialize_catalog()


for aoi in aois:
    print("\n---------" + aoi["title"] + "---------\n")
    search_paramaters = catalog.construct_parameters(geometry=aoi['geometry'], 
                                                     start_date="2019-01-01",
                                                     end_date="2020-12-31",
                                                     sensors=["spot"],
                                                     max_cloudcover=5,
                                                     sortby="acquisitionDate",
                                                     ascending=False,
                                                     limit=3)
    search_results = catalog.search(search_paramaters)
    
    # Download & Visualise quicklooks
    catalog.download_quicklooks(image_ids=search_results.id.to_list(), sensor="spot")
    display(search_results.head())
    catalog.plot_quicklooks(figsize=(18,5), titles=search_results.sceneId.to_list())

    # Select least cloud scene for further workflow
    aoi["sceneId"] = search_results.sceneId.to_list()[0]
catalog = up42.initialize_catalog()


for aoi in aois:
    print("\n---------" + aoi["title"] + "---------\n")
    search_paramaters = catalog.construct_parameters(geometry=aoi['geometry'], 
                                                     start_date="2019-01-01",
                                                     end_date="2020-12-31",
                                                     sensors=["spot"],
                                                     max_cloudcover=5,
                                                     sortby="acquisitionDate",
                                                     ascending=False,
                                                     limit=3)
    search_results = catalog.search(search_paramaters)
    
    # Download & Visualise quicklooks
    catalog.download_quicklooks(image_ids=search_results.id.to_list(), sensor="spot")
    display(search_results.head())
    catalog.plot_quicklooks(figsize=(18,5), titles=search_results.sceneId.to_list())

    # Select least cloud scene for further workflow
    aoi["sceneId"] = search_results.sceneId.to_list()[0]

---------SBY---------

100%|██████████| 3/3 [00:02<00:00,  1.50it/s]

	geometry	id	acquisitionDate	constellation	providerName	blockNames	cloudCoverage	up42:usageType	providerProperties	SceneId
0	POLYGON ((112.56527 -6.78011, 112.56507 -7.320...	53341a16-7294-4cfd-bcc8-de40cace639f	2020-09-29T02:20:15Z	SPOT	oneatlas	[oneatlas-spot-fullscene, oneatlas-spot-aoicli...	1.22655	[DATA, ANALYTICS]	{'acquisitionDate': '2020-09-29T02:20:15.624Z'...	DS_SPOT6_202009290220156_FR1_FR1_FR1_FR1_E113S...
1	POLYGON ((112.58667 -6.76592, 112.58758 -7.301...	66c48450-288e-4e19-8829-b44c121a8e79	2019-11-09T02:22:38Z	SPOT	oneatlas	[oneatlas-spot-fullscene, oneatlas-spot-aoicli...	2.31000	[DATA, ANALYTICS]	{'commercialReference': 'SO19050079', 'acquisi...	DS_SPOT7_201911090222385_FR1_FR1_SV1_SV1_E113S...
2	POLYGON ((112.54476 -6.75036, 113.19278 -6.751...	94d9ce84-c35c-467a-8732-3a9bfbe725b7	2019-05-25T02:12:44Z	SPOT	oneatlas	[oneatlas-spot-fullscene, oneatlas-spot-aoicli...	4.08000	[DATA, ANALYTICS]	{'commercialReference': 'SO19018464', 'acquisi...	DS_SPOT7_201905250212445_FR1_FR1_SV1_SV1_E113S...