Workflow¶
Analytics platform discontinued after January 31, 2024
The current analytics platform will be discontinued after January 31, 2024, and will be replaced by new advanced processing functionalities. This change will affect projects, workflows, jobs, data blocks, processing blocks, and custom blocks. For more information, see the blog post.
The Workflow class enables access to the UP42 analytics functionality.
A workflow is a sequence of data blocks and processing blocks. It defines an order of operations that start with a data block, which may be followed by up to five processing blocks.
workflow = up42.initialize_workflow(
project_id="68567134-27ad-7bd7-4b65-d61adb11fc78",
workflow_id="7fb2ec8a-45be-41ad-a50f-98ba6b528b98",
)
Projects¶
max_concurrent_jobs¶
The max_concurrent_jobs
attribute returns the maximum number of jobs that can run simultaneously.
The returned format is int
.
Example
workflow.max_concurrent_jobs
Workflows¶
info¶
The info
attribute returns metadata of a specific workflow.
The returned format is dict
.
Example
workflow.info
update_name()¶
The update_name()
function allows you to change the name and description of a workflow.
update_name(
name,
description,
)
Arguments
Argument | Overview |
---|---|
name |
str The workflow name. |
description |
str The workflow description. |
Example
workflow.update_name(
name="Sentinel-2 with tiling",
description="Implements tiling on free Sentinel-2 data",
)
delete()¶
The delete()
function allows you to delete a workflow.
delete()
Example
workflow.delete()
Workflow tasks¶
Workflow tasks are blocks that are added to a workflow. A workflow task uses a specific block version that specifies its input JSON parameters and what blocks can be added before and after it.
workflow_tasks¶
The workflow_tasks
attribute returns a list of workflow tasks in a workflow.
The returned format is dict[str, str]
.
Example
workflow.workflow_tasks
get_workflow_tasks()¶
The get_workflow_tasks
function returns a list of workflow tasks in a workflow.
get_workflow_tasks(basic)
The returned format is Union[list, dict]
.
Arguments
Argument | Overview |
---|---|
basic |
bool Determines how to return a list of workflow tasks:
False . |
Example
workflow.get_workflow_tasks(basic=True)
add_workflow_tasks()¶
The add_workflow_tasks()
function allows you to add workflow tasks to a workflow. Include the full sequence of selected workflow tasks as the function overwrites existing workflow tasks.
add_workflow_tasks(input_tasks)
Arguments
Argument | Overview |
---|---|
input_tasks |
Union[list[str], list[dict]] / required The workflow tasks. To specify a specific version of a block, use block IDs. To specify the most recent version of a block, use block names or block display names. |
Example
workflow.add_workflow_tasks(
input_tasks=["sentinelhub-s2-aoiclipped", "tiling"],
)
get_compatible_blocks()¶
The get_compatible_blocks()
function returns a list of compatible blocks that can be added after the last workflow task in a workflow.
If there are no workflow tasks, it will return a list of all available data blocks.
get_compatible_blocks()
The returned format is dict
.
Example
workflow.get_compatible_blocks()
get_parameters_info()¶
The get_parameters_info()
function returns the input JSON parameters of each workflow task.
get_parameters_info()
The returned format is dict
.
Example
workflow.get_parameters_info()
Jobs¶
construct_parameters()¶
The construct_parameters()
function allows you to fill out the JSON parameters for a job.
construct_parameters(
geometry,
geometry_operation,
start_date,
end_date,
limit,
scene_ids,
asset_ids,
)
The returned format is dict
.
Arguments
Argument | Overview |
---|---|
geometry |
Union[FeatureCollection, Feature, dict, list, GeoDataFrame, Polygon, geojson_Polygon] The geometry of interest. |
geometry_operation |
str The geometric filter. The allowed values:
|
start_date |
Union[str, datetime] The start date of the search period in the YYYY-MM-DD or YYYY-MM-DDTHH:MM:SS format. |
end_date |
Union[str, datetime] The end date of the search period in the YYYY-MM-DD or YYYY-MM-DDTHH:MM:SS format. |
limit |
int The maximum number of expected results. |
scene_ids |
list[str] The scene IDs. If used, all other parameters except geometry are ignored. |
asset_ids |
list[str] The asset IDs. Use with Processing from Storage block. |
Example
workflow.construct_parameters(
geometry=up42.get_example_aoi(location="Berlin"),
geometry_operation="bbox",
start_date="2020-01-01",
end_date="2022-12-31",
limit=1,
)
construct_parameters_parallel()¶
The construct_parameters()
function allows you to fill out the JSON parameters for multiple jobs to run in parallel.
construct_parameters_parallel(
geometries,
interval_dates,
scene_ids,
limit_per_job,
geometry_operation,
)
The returned format is list[dict]
.
Arguments
Argument | Overview |
---|---|
geometries |
list[Union[dict, Feature, geojson_Polygon, Polygon]] The geometries of interest. |
interval_dates |
list[tuple[str, str]] The start and end dates in the YYYY-MM-DD format. |
scene_ids |
list[str] The scene IDs. If used, all other parameters except geometry are ignored. |
limit_per_job |
int The maximum number of expected results per job. The default value is 1 . |
geometry_operation |
str The geometric filter. The allowed values:
intersects . |
Example
workflow.construct_parameters_parallel(
geometries=[up42.get_example_aoi(location="Berlin")],
interval_dates=[("2023-01-01", "2023-01-30"), ("2023-02-01", "2023-02-26")],
limit_per_job=2,
geometry_operation="bbox",
)
estimate_job()¶
The estimate_job()
returns the cost estimate for a job.
estimate_job(input_parameters)
The returned format is dict
.
Arguments
Argument | Overview |
---|---|
input_parameters |
Union[dict, str, Path] The job JSON parameters. |
Example
workflow.estimate_job(
input_parameters=workflow.construct_parameters(
geometry=up42.get_example_aoi(location="Berlin"),
geometry_operation="bbox",
start_date="2020-01-01",
end_date="2022-12-31",
limit=1,
),
)
get_jobs()¶
The get_jobs()
function returns all jobs associated with a workflow.
get_jobs(
return_json,
test_jobs,
real_jobs,
)
The returned format is Union[JobCollection, list[dict]]
.
Arguments
Argument | Overview |
---|---|
return_json |
bool Determines how to return jobs:
False . |
test_jobs |
bool Determines whether to return test queries:
True . |
real_jobs |
bool Determines whether to return live jobs:
True . |
Example
workflow.get_jobs(
return_json=True,
test_jobs=False,
real_jobs=True,
)
test_job()¶
The test_job()
function allows you to create a test query and run it.
test_job(
input_parameters,
track_status,
name,
)
The returned format is Job
.
Arguments
Argument | Overview |
---|---|
input_parameters |
Union[dict, str, Path] The job JSON parameters. |
track_status |
bool Determines whether to query job status every 30 seconds:
False . |
name |
str The job name. By default, the workflow name is used. |
Example
# Construct job JSON parameters
input_parameters=workflow.construct_parameters(
geometry=up42.get_example_aoi(location="Berlin"),
geometry_operation="bbox",
start_date="2020-01-01",
end_date="2022-12-31",
limit=1,
)
# Run test query
workflow.test_job(
input_parameters=input_parameters,
track_status=True,
name="Test Job 1",
)
test_jobs_parallel()¶
The test_jobs_parallel()
function allows you to create multiple test queries and run them in parallel.
test_jobs_parallel(
input_parameters_list,
name,
max_concurrent_jobs,
)
The returned format is JobCollection
.
Arguments
Argument | Overview |
---|---|
input_parameters_list |
list[dict] The parallel jobs' JSON parameters. |
name |
str The prefix for the job names. By default, the workflow name is used. |
max_concurrent_jobs |
int The maximum number of jobs that can run simultaneously. The default value is 10 . |
Example
# Construct parallel jobs' JSON parameters
input_parameters_list=workflow.construct_parameters_parallel(
geometries=[up42.get_example_aoi(location="Berlin")],
interval_dates=[("2023-01-01", "2023-01-30"), ("2023-02-01", "2023-02-26")],
limit_per_job=2,
geometry_operation="bbox",
)
# Run test queries
workflow.test_jobs_parallel(
input_parameters_list=input_parameters_list,
name="Test Job",
max_concurrent_jobs=5,
)
run_job()¶
The run_job()
function allows you to create a job and run it.
run_job(
input_parameters,
track_status,
name,
)
The returned format is Job
.
Arguments
Argument | Overview |
---|---|
input_parameters |
Union[dict, str, Path] The job JSON parameters. |
track_status |
bool Determines whether to query job status every 30 seconds:
False . |
name |
str The job name. By default, the workflow name is used. |
Example
# Construct job JSON parameters
input_parameters=workflow.construct_parameters(
geometry=up42.get_example_aoi(location="Berlin"),
geometry_operation="bbox",
start_date="2020-01-01",
end_date="2022-12-31",
limit=1,
)
# Run job
workflow.run_jobs(
input_parameters=input_parameters,
track_status=True,
name="Processing workflow",
)
run_jobs_parallel()¶
The run_jobs_parallel()
function allows you to create multiple jobs and run them in parallel.
run_jobs_parallel(
input_parameters_list,
name,
max_concurrent_jobs,
)
The returned format is JobCollection
.
Arguments
Argument | Overview |
---|---|
input_parameters_list |
list[dict] The parallel jobs' JSON parameters. |
name |
str The prefix for the job names. By default, the workflow name is used. |
max_concurrent_jobs |
int The maximum number of jobs that can run simultaneously. The default value is 10 . |
Example
# Construct parallel jobs' JSON parameters
input_parameters_list=workflow.construct_parameters_parallel(
geometries=[up42.get_example_aoi(location="Berlin")],
interval_dates=[("2023-01-01", "2023-01-30"), ("2023-02-01", "2023-02-26")],
limit_per_job=2,
geometry_operation="bbox",
)
# Run jobs
workflow.run_jobs_parallel(
input_parameters_list=input_parameters_list,
name="Processing workflow",
max_concurrent_jobs=5,
)