Skip to content
This repository has been archived by the owner on Apr 26, 2024. It is now read-only.

Add support for JSON patch customizations of the ECSTask task run payload #120

Merged
merged 3 commits into from
Oct 4, 2022

Conversation

zanieb
Copy link
Contributor

@zanieb zanieb commented Sep 30, 2022

Summary

This exposes customizations of the payload for ECS task runs, allowing users to set fields not available at the top level or override any of the options that Prefect sets.

We follow the JSON Patch customizations pattern used in the core library for Kubernetes. Tests and examples have been added for the current user requests. Future requests can be followed with new documentation examples, as patches can be challenging to write at times.

Relevant Issue(s)

Closes #115
Closes #113
Closes #112

Checklist

  • Summarized PR's changes in the Unreleased section of the CHANGELOG.md

@zanieb
Copy link
Contributor Author

zanieb commented Oct 3, 2022

Note I have not done QA of this feature against a live ECS cluster. cc @anna-geller ?

Copy link
Contributor

@anna-geller anna-geller left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Works and looks great in the UI:

image

The subnet matches
image

from prefect_aws.ecs import ECSTask, AwsCredentials

aws_credentials = AwsCredentials.load("prod")

ecs = ECSTask(
    # command=["echo", "hello ECS"],
    aws_credentials=aws_credentials,
    image="338306982838.dkr.ecr.us-east-1.amazonaws.com/dataflowops:a38085ff8170eb04db3a0ad14247558362105ea4",
    # image="prefecthq/prefect:2-python3.9",
    cpu="256",
    memory="512",
    stream_output=True,
    configure_cloudwatch_logs=True,
    cluster="prefect",
    execution_role_arn="arn:aws:iam::338306982838:role/dataflowops_ecs_execution_role",
    task_role_arn="arn:aws:iam::338306982838:role/dataflowops_ecs_task_role",
    task_start_timeout_seconds=90,
    vpc_id="vpc-0ff32ab58b1c8695a",
    task_customizations=[
        {
            "op": "add",
            "path": "/networkConfiguration/awsvpcConfiguration/subnets",
            "value": ["subnet-052624e5d692fda86"],
        },
    ],
)
ecs.save("customized", overwrite=True)

@zanieb zanieb merged commit d73b396 into main Oct 4, 2022
@zanieb zanieb deleted the add-customizations-to-task-runs branch October 4, 2022 14:11
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
3 participants