Skip to content

Example: Radar processing

The processing of radar data - often referred to as SAR data, which stands for Synthetice Aperture Radar - is a topic of its own and quite different from optical data that uses Red, Green, Blue, NIR and similar bands. This is the first of a series of jupyter notebooks where example SAR workflows will be explained that can be run on UP42. All of the examples are based on a specific type of analysis which is called polarimetry and processes one image at a time. In comparison to polarimetric there also is interferometric analysis which leverages the information encoded in suitable pairs of radar image.

In this notebook we want to have a look at the Sentinel-1 GRD full scene block, followed by SNAP Sentinel-1 Polarimetric Processing. Polarimetric SAR Processing turns a Sentinel-1 GRD image into a multichannel GeoTIFF that is ready for analysis.

  • Prepare workflow
  • Define aoi and input parameters
  • Run Job
  • Visualize results
In [1]:
import up42
In [3]:
# Authenticate with UP42
up42.authenticate(cfg_file="config.json")
#up42.authenticate(project_id=12345, project_api_key=12345)
2020-05-26 14:19:56,546 - up42.auth - INFO - Got credentials from config file.
2020-05-26 14:19:57,155 - up42.auth - INFO - Authentication with UP42 successful!

Prepare UP42 workflows

Create a new project on UP42 or use an existing one.

In [5]:
S1_SNAP_project = up42.initialize_project()
2020-05-26 14:20:10,960 - up42 - INFO - Working on Project with project_id d3ea9123-2b89-4975-be16-e978fed8329c

Create workflow and check available blocks and data

In [6]:
workflow = S1_SNAP_project.create_workflow(name="S1-GRD_SNAP", 
                                   use_existing=True)
print(up42.get_blocks(basic=True))
2020-05-26 14:20:14,765 - up42.project - INFO - Getting existing workflows in project ...
2020-05-26 14:20:15,434 - up42.project - INFO - Got 12 workflows for project d3ea9123-2b89-4975-be16-e978fed8329c.
100%|██████████| 12/12 [00:07<00:00,  1.52it/s]
2020-05-26 14:20:23,341 - up42.project - INFO - Using existing workflow: S1-GRD_SNAP, 1585a205-3907-4905-bd02-057a17c3cf84.
2020-05-26 14:20:24,728 - up42.tools - INFO - Getting blocks name and id, use basic=False for all block details.
{'tiling': 'd350aa0b-ac31-4021-bbe6-fd8da366740a', 'oneatlas-spot-aoiclipped': '0f15e07f-efcc-4598-939b-18aade349c57', 'oneatlas-pleiades-aoiclipped': 'f026874d-e95e-4293-b811-7667130e054d', 'sobloo-s1-grd-fullscene': '4524e2de-c780-488d-9818-fe68dad9f095', 'sobloo-s2-l1c-fullscene': '604988cb-8252-4161-bf28-f6fb63d7371c', 'snap-polarimetric': '320158d6-8f93-4484-a828-e1fb64f677ff', 'sentinelhub-s2-aoiclipped': 'c4758545-4b74-4318-ae1f-d5ba72f234ca', 'sentinelhub-landsat8-aoiclipped': 'e0b133ae-7b9c-435c-99ac-c4527cc8d9cf', 'sobloo-s1-slc-fullscene': 'cf822545-c73c-467b-8f43-5311dbefe03f', 'nasa-modis': '61279eb8-02e1-4b7a-ac3d-1f62d19d3484', 's2-superresolution': '4872fef8-aec8-4dec-adcb-560ee4430a2b', 'oneatlas-pleiades-fullscene': '8487adcd-a4d7-4cb7-b826-75a533e1f330', 'oneatlas-spot-fullscene': 'aa62113f-0dd1-40a3-a004-954c9d087071', 'data-conversion': '470eedda-5f62-433c-8562-98eb8783af87', 'pansharpen': '2f24c662-c129-409f-a7c3-afa16a4c78cb', 'sobloo-s1-grd-aoiclipped': 'a956166f-c0ed-4670-8a43-87bed8d222f3', 'hello-world': '77899bd0-40de-4aa7-b67e-513a5655afcb', 'ndvi': 'aecb81ef-1c92-4b2e-aa25-55ccebe2f90a', 'sobloo-s2-l1c-aoiclipped': 'a2daaab4-196d-4226-a018-a810444dcad1', 'sobloo-s3': 'fee13ec1-a067-4d6a-95dc-a4fef458f4f4', 'sobloo-s5p': 'cba7c59b-548d-48bf-8920-c20d7d316dfd', 'kmeans-clustering': 'adf21e0a-98bf-41a9-a2cc-a59898a461ba', 'vectorising': '295bf286-0748-474f-aa38-c1ac35151204', 'crs-conversion': '18e6772f-cf33-4955-b7e4-61df8a108fd9', 'sharpening': '4ed70368-d4e1-4462-bef6-14e768049471', 'zonal-statistics': 'a5b8b938-6fd6-4bac-92cd-dffd7f3169aa', 'superresolution': '6c914299-7203-40ad-9b89-de0b4e827bd0', 's5p-lvl3': '9a593e06-eca0-49e0-8b8c-6fe95f699f9d', 'hexagon-aerial-30cm': '0b04d9f7-3a8e-4467-9fb8-1e8343c9469a', 'hexagon-aerial-15cm': '36fe7d3a-4671-424b-bd54-918dd21cfde1', 'ship-identification': '20a3bd1e-3f27-40cf-9915-8fa3d5024ade', 'ais-hvp': '7394287a-2458-4204-be62-36b6d264bcfe', 'ais-hvt': '67eb1763-abeb-4188-b135-f6a0d669d759', 'meteomatics': 'ed0beedb-111b-4285-aa2d-f876a4c16a32', 'oneatlas-pleiades-primary': 'd1e5e0de-71fa-4488-9c0e-3f22ac74a2b6', 'tiled-k-means': '45c0284a-5cd7-4bc0-9d6c-db05f7271036', 'data-conversion-netcdf': 'c358d5af-7819-4ecf-b8b3-629d5d3ba319', 'data-conversion-dimap': '25f42430-d108-4ea4-a81d-2c2b3fff7d11', 'aws-s2-l2a': '73f1b9e0-83de-4cbd-addf-5e0d23ed65d4', 'nextmapone-3m': '56a0ac08-69a5-491a-993d-63e3b9da315c', 'nextmapone-6m': 'f87dd9f3-fc14-48b1-b77d-d695fb65fcc4', 'nextmapone-1m': 'bfd43fbc-b662-4874-9147-658a55bf9edc'}
In [7]:
# Fill the workflow with tasks
input_tasks= ['sobloo-s1-grd-fullscene', 'snap-polarimetric']
workflow.add_workflow_tasks(input_tasks=input_tasks)
workflow.get_parameters_info()
2020-05-26 14:21:03,273 - up42.workflow - INFO - Added tasks to workflow: [{'name': 'sobloo-s1-grd-fullscene:1', 'parentName': None, 'blockId': '4524e2de-c780-488d-9818-fe68dad9f095'}, {'name': 'snap-polarimetric:1', 'parentName': 'sobloo-s1-grd-fullscene:1', 'blockId': '320158d6-8f93-4484-a828-e1fb64f677ff'}]
2020-05-26 14:21:04,000 - up42.workflow - INFO - Got 2 tasks/blocks in workflow 1585a205-3907-4905-bd02-057a17c3cf84.
Out[7]:
{'sobloo-s1-grd-fullscene:1': {'ids': {'type': 'array', 'default': None},
  'bbox': {'type': 'array', 'default': None},
  'time': {'type': 'dateRange',
   'default': '2018-01-01T00:00:00+00:00/2020-12-31T23:59:59+00:00'},
  'limit': {'type': 'integer', 'default': 1, 'minimum': 1},
  'contains': {'type': 'geometry'},
  'intersects': {'type': 'geometry'},
  'time_series': {'type': 'array', 'default': None},
  'orbit_direction': {'type': 'string', 'default': None},
  'acquisition_mode': {'type': 'string', 'default': None}},
 'snap-polarimetric:1': {'bbox': {'type': 'array', 'default': None},
  'mask': {'type': 'array',
   'items': {'enum': ['land', 'sea'], 'type': 'string'},
   'default': None},
  'contains': {'type': 'geometry', 'default': None},
  'intersects': {'type': 'geometry', 'default': None},
  'clip_to_aoi': {'type': 'boolean', 'default': False},
  'tcorrection': {'type': 'boolean', 'default': True},
  'linear_to_db': {'type': 'boolean', 'default': True},
  'polarisations': {'type': 'array',
   'items': {'enum': ['VV', 'VH'], 'type': 'string'},
   'default': ['VV'],
   'required': False,
   'description': 'Requested polarisations for the output'},
  'speckle_filter': {'type': 'boolean', 'default': True},
  'calibration_band': {'type': 'array', 'default': ['sigma']}}}

Define aoi and input parameters

The S1 GRD block always delivers the complete images as they are delivered in SAFE format which cannot be clipped. The image can be clipped though by the SNAP polarimetric processing blocks. For this it is necessary to supply the same geometry (in this case a bbox) to that block as well.

In [8]:
input_parameters = {
  "sobloo-s1-grd-fullscene:1": {
    "bbox": [13.371037, 52.512799, 13.382624, 52.518747],
    "ids": None,
    "time": "2018-01-01T00:00:00+00:00/2020-12-31T23:59:59+00:00",
    "limit": 1,
    "zoom_level": 14,
  },
    "snap-polarimetric:1": {
    "bbox": [13.371037, 52.512799, 13.382624, 52.518747],
    "mask": None,
    "contains": None,
    "intersects": None,
    "clip_to_aoi": True,
    "tcorrection": True,
    "linear_to_db": True,
    "polarisations": [
        "VV"
   ],
   "speckle_filter": True,
   "calibration_band": [
         "sigma"
   ]
  }
}

Run test query

Run a test job to query data availability and check the configuration.

In [9]:
test_job = workflow.test_job(input_parameters=input_parameters, track_status=True)
test_results = test_job.get_results_json()
print(test_results)
2020-05-26 14:21:14,155 - up42.workflow - INFO - +++++++++++++++++++++++++++++++++
2020-05-26 14:21:14,157 - up42.workflow - INFO - Running this job as Test Query...
2020-05-26 14:21:14,157 - up42.workflow - INFO - +++++++++++++++++++++++++++++++++
2020-05-26 14:21:14,158 - up42.workflow - INFO - Selected input_parameters: {'sobloo-s1-grd-fullscene:1': {'bbox': [13.371037, 52.512799, 13.382624, 52.518747], 'ids': None, 'time': '2018-01-01T00:00:00+00:00/2020-12-31T23:59:59+00:00', 'limit': 1, 'zoom_level': 14}, 'snap-polarimetric:1': {'bbox': [13.371037, 52.512799, 13.382624, 52.518747], 'mask': None, 'contains': None, 'intersects': None, 'clip_to_aoi': True, 'tcorrection': True, 'linear_to_db': True, 'polarisations': ['VV'], 'speckle_filter': True, 'calibration_band': ['sigma']}, 'config': {'mode': 'DRY_RUN'}}.
2020-05-26 14:21:15,659 - up42.workflow - INFO - Created and running new job: 28c72c74-a5dc-46e3-bb70-f6efad3f910c.
2020-05-26 14:21:16,144 - up42.job - INFO - Tracking job status continuously, reporting every 30 seconds...
2020-05-26 14:21:39,235 - up42.job - INFO - Job finished successfully! - 28c72c74-a5dc-46e3-bb70-f6efad3f910c
2020-05-26 14:21:44,956 - up42.job - INFO - Retrieved 1 features.
{'type': 'FeatureCollection', 'features': [{'type': 'Feature', 'bbox': [12.306028, 52.168652, 16.66515, 54.078037], 'id': '5a0bcc6b-8316-45c9-9106-8de8e27e5753', 'geometry': {'type': 'Polygon', 'coordinates': [[[16.151102, 52.168652], [16.66515, 53.660217], [12.685362, 54.078037], [12.306028, 52.5835], [16.151102, 52.168652]]]}, 'properties': {'attachments': [], 'visibility': {'deleted': False}, 'illumination': {}, 'production': {'levelCode': 'L1', 'ongoing': False, 'timeliness': 'Fast-24h'}, 'archive': {'offLine': False, 'filename': 'S1A_IW_GRDH_1SDV_20200525T051659_20200525T051724_032717_03CA2F_DB50.SAFE', 'size': 1689, 'format': 'SAFE', 'onLine': False}, 'spatialCoverage': {'verticality': {}, 'geometry': {'geographicBoundingPolygon': {'coordinates': [[[16.151102, 52.168652], [16.66515, 53.660217], [12.685362, 54.078037], [12.306028, 52.5835], [16.151102, 52.168652]]], 'type': 'Polygon'}, 'global': False, 'centerPoint': {'lon': 14.45556570632881, 'lat': 53.12652491313295}}}, 'quality': {'qualified': False}, 'target': {}, 'timeStamp': 1590383819131, 'uid': '5a0bcc6b-8316-45c9-9106-8de8e27e5753', 'identification': {'profile': 'Image', 'externalId': 'S1A_IW_GRDH_1SDV_20200525T051659_20200525T051724_032717_03CA2F_DB50', 'collection': 'Sentinel-1', 'type': 'GRD', 'dataset': {}}, 'transmission': {}, 'contentDescription': {}, 'provider': {}, 'acquisition': {'endViewingDate': 1590383844129, 'mission': 'Sentinel-1', 'missionId': 'A', 'missionCode': 'S1A', 'beginViewingDate': 1590383819131, 'missionName': 'Sentinel-1A', 'polarization': 'VV VH', 'type': 'NOMINAL', 'sensorMode': 'IW', 'sensorId': 'SAR-C SAR'}, 'orbit': {'relativeNumber': 95, 'lastRelativeNumber': 95, 'lastNumber': 32717, 'direction': 'DESCENDING'}, 'state': {'resources': {'thumbnail': True, 'quicklook': True}, 'services': {'wmts': False, 'download': 'internal', 'wcs': False, 'wms': False}, 'insertionDate': 1590395388355}, 'attitude': {}}}]}

Run the job

In [10]:
job = workflow.run_job(input_parameters=input_parameters, track_status=True)
2020-05-26 14:21:44,963 - up42.workflow - INFO - Selected input_parameters: {'sobloo-s1-grd-fullscene:1': {'bbox': [13.371037, 52.512799, 13.382624, 52.518747], 'ids': None, 'time': '2018-01-01T00:00:00+00:00/2020-12-31T23:59:59+00:00', 'limit': 1, 'zoom_level': 14}, 'snap-polarimetric:1': {'bbox': [13.371037, 52.512799, 13.382624, 52.518747], 'mask': None, 'contains': None, 'intersects': None, 'clip_to_aoi': True, 'tcorrection': True, 'linear_to_db': True, 'polarisations': ['VV'], 'speckle_filter': True, 'calibration_band': ['sigma']}}.
2020-05-26 14:21:46,310 - up42.workflow - INFO - Created and running new job: 2e0e9cf0-8d80-439a-bc50-179543641ea9.
2020-05-26 14:21:46,734 - up42.job - INFO - Tracking job status continuously, reporting every 30 seconds...
2020-05-26 14:22:21,006 - up42.job - INFO - Job is RUNNING! - 2e0e9cf0-8d80-439a-bc50-179543641ea9
2020-05-26 14:23:06,480 - up42.job - INFO - Job is RUNNING! - 2e0e9cf0-8d80-439a-bc50-179543641ea9
2020-05-26 14:23:40,185 - up42.job - INFO - Job is RUNNING! - 2e0e9cf0-8d80-439a-bc50-179543641ea9
2020-05-26 14:24:13,848 - up42.job - INFO - Job is RUNNING! - 2e0e9cf0-8d80-439a-bc50-179543641ea9
2020-05-26 14:24:47,455 - up42.job - INFO - Job is RUNNING! - 2e0e9cf0-8d80-439a-bc50-179543641ea9
2020-05-26 14:25:21,147 - up42.job - INFO - Job is RUNNING! - 2e0e9cf0-8d80-439a-bc50-179543641ea9
2020-05-26 14:25:54,825 - up42.job - INFO - Job is RUNNING! - 2e0e9cf0-8d80-439a-bc50-179543641ea9
2020-05-26 14:26:28,527 - up42.job - INFO - Job is RUNNING! - 2e0e9cf0-8d80-439a-bc50-179543641ea9
2020-05-26 14:27:02,484 - up42.job - INFO - Job is RUNNING! - 2e0e9cf0-8d80-439a-bc50-179543641ea9
2020-05-26 14:27:36,284 - up42.job - INFO - Job is RUNNING! - 2e0e9cf0-8d80-439a-bc50-179543641ea9
2020-05-26 14:28:09,935 - up42.job - INFO - Job is RUNNING! - 2e0e9cf0-8d80-439a-bc50-179543641ea9
2020-05-26 14:28:42,827 - up42.job - INFO - Job is RUNNING! - 2e0e9cf0-8d80-439a-bc50-179543641ea9
2020-05-26 14:29:15,409 - up42.job - INFO - Job is RUNNING! - 2e0e9cf0-8d80-439a-bc50-179543641ea9
2020-05-26 14:29:48,015 - up42.job - INFO - Job is RUNNING! - 2e0e9cf0-8d80-439a-bc50-179543641ea9
2020-05-26 14:30:20,694 - up42.job - INFO - Job finished successfully! - 2e0e9cf0-8d80-439a-bc50-179543641ea9

Visualize the results

In [12]:
# Download results:
job.download_results()

# Visualize downloaded results
job.plot_results()

Next processing steps

The output of the workflow is a GeoTIFF file which can be processed by different algorithms. It is for example possible to apply raster tiling and then do some machine learning based analysis at the end.