-
Notifications
You must be signed in to change notification settings - Fork 1.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[ADXT-199] Early start long e2e tests environment deployment on the CI (priority on EKS) #26891
base: main
Are you sure you want to change the base?
Conversation
Test changes on VMUse this command from test-infra-definitions to manually test this PR changes on a VM: inv create-vm --pipeline-id=46300838 --os-family=ubuntu Note: This applies to commit 4f173fc |
…/test-early-create-eks
Regression DetectorRegression Detector ResultsRun ID: b41ed032-d8c5-43fd-baf7-e50fa8ef8fbd Metrics dashboard Target profiles Baseline: 304623b Performance changes are noted in the perf column of each table:
No significant changes in experiment optimization goalsConfidence level: 90.00% There were no significant changes in experiment optimization goals at this confidence level and effect size tolerance.
|
perf | experiment | goal | Δ mean % | Δ mean % CI | trials | links |
---|---|---|---|---|---|---|
➖ | uds_dogstatsd_to_api_cpu | % cpu utilization | +1.67 | [+0.95, +2.40] | 1 | Logs |
➖ | basic_py_check | % cpu utilization | +0.64 | [-3.28, +4.57] | 1 | Logs |
➖ | file_to_blackhole_1000ms_latency | egress throughput | +0.25 | [-0.25, +0.75] | 1 | Logs |
➖ | file_to_blackhole_300ms_latency | egress throughput | +0.15 | [-0.04, +0.33] | 1 | Logs |
➖ | file_to_blackhole_500ms_latency | egress throughput | +0.04 | [-0.20, +0.28] | 1 | Logs |
➖ | uds_dogstatsd_to_api | ingress throughput | +0.01 | [-0.09, +0.11] | 1 | Logs |
➖ | file_to_blackhole_100ms_latency | egress throughput | +0.01 | [-0.22, +0.23] | 1 | Logs |
➖ | tcp_dd_logs_filter_exclude | ingress throughput | -0.00 | [-0.02, +0.01] | 1 | Logs |
➖ | otel_to_otel_logs | ingress throughput | -0.01 | [-0.82, +0.80] | 1 | Logs |
➖ | file_to_blackhole_0ms_latency | egress throughput | -0.03 | [-0.35, +0.30] | 1 | Logs |
➖ | pycheck_lots_of_tags | % cpu utilization | -0.04 | [-3.49, +3.41] | 1 | Logs |
➖ | idle | memory utilization | -0.15 | [-0.21, -0.09] | 1 | Logs |
➖ | file_tree | memory utilization | -1.26 | [-1.41, -1.12] | 1 | Logs |
➖ | tcp_syslog_to_blackhole | ingress throughput | -1.70 | [-1.77, -1.63] | 1 | Logs |
➖ | idle_all_features | memory utilization | -2.41 | [-2.56, -2.27] | 1 | Logs |
Bounds Checks
perf | experiment | bounds_check_name | replicates_passed |
---|---|---|---|
✅ | file_to_blackhole_0ms_latency | memory_usage | 10/10 |
✅ | file_to_blackhole_1000ms_latency | memory_usage | 10/10 |
✅ | file_to_blackhole_100ms_latency | memory_usage | 10/10 |
✅ | file_to_blackhole_300ms_latency | memory_usage | 10/10 |
✅ | file_to_blackhole_500ms_latency | memory_usage | 10/10 |
✅ | idle | memory_usage | 10/10 |
Explanation
A regression test is an A/B test of target performance in a repeatable rig, where "performance" is measured as "comparison variant minus baseline variant" for an optimization goal (e.g., ingress throughput). Due to intrinsic variability in measuring that goal, we can only estimate its mean value for each experiment; we report uncertainty in that value as a 90.00% confidence interval denoted "Δ mean % CI".
For each experiment, we decide whether a change in performance is a "regression" -- a change worth investigating further -- if all of the following criteria are true:
-
Its estimated |Δ mean %| ≥ 5.00%, indicating the change is big enough to merit a closer look.
-
Its 90.00% confidence interval "Δ mean % CI" does not contain zero, indicating that if our statistical model is accurate, there is at least a 90.00% chance there is a difference in performance between baseline and comparison variants.
-
Its configuration does not mark it "erratic".
@vboulineau would you consider migrating to |
Gitlab CI Configuration ChangesModified Jobsstages (configuration) stages:
- .pre
- setup
- maintenance_jobs
- deps_build
- deps_fetch
- lint
- source_test
- source_test_stats
- software_composition_analysis
- binary_build
- package_deps_build
- kernel_matrix_testing_prepare
- kernel_matrix_testing_system_probe
- kernel_matrix_testing_security_agent
- kernel_matrix_testing_cleanup
- integration_test
- benchmarks
- package_build
- packaging
- pkg_metrics
- kitchen_deploy
- kitchen_testing
- container_build
- container_scan
- check_deploy
- dev_container_deploy
- deploy_containers
- deploy_packages
- deploy_cws_instrumentation
- deploy_dca
- choco_and_install_script_build
- trigger_release
- choco_and_install_script_deploy
- internal_image_deploy
- install_script_testing
- e2e_pre_test
+ - e2e_init
- e2e
+ - e2e_cleanup
- e2e_k8s
- e2e_install_packages
- kitchen_cleanup
- functional_test
- functional_test_cleanup
- junit_upload
- internal_kubernetes_deploy
- post_rc_build
- check_merge
- notify
- .post variables (configuration) variables:
AGENT_BINARIES_DIR: bin/agent
AGENT_GITHUB_APP: agent-github-app
AGENT_GITHUB_APP_ID: ci.datadog-agent.platform-github-app-id
AGENT_GITHUB_INSTALLATION_ID: ci.datadog-agent.platform-github-app-installation-id
AGENT_GITHUB_KEY: ci.datadog-agent.platform-github-app-key
AGENT_QA_PROFILE: ci.datadog-agent.agent-qa-profile
API_KEY_DDDEV: ci.datadog-agent.datadog_api_key
API_KEY_ORG2: ci.datadog-agent.datadog_api_key_org2
APP_KEY_ORG2: ci.datadog-agent.datadog_app_key_org2
ARTIFACT_DOWNLOAD_ATTEMPTS: 2
ATLASSIAN_WRITE: atlassian-write
BTFHUB_ARCHIVE_BRANCH: main
BUCKET_BRANCH: dev
CHANGELOG_COMMIT_SHA: ci.datadog-agent.gitlab_changelog_commit_sha
CHOCOLATEY_API_KEY: ci.datadog-agent.chocolatey_api_key
CLANG_LLVM_VER: 12.0.1
CLUSTER_AGENT_BINARIES_DIR: bin/datadog-cluster-agent
CLUSTER_AGENT_CLOUDFOUNDRY_BINARIES_DIR: bin/datadog-cluster-agent-cloudfoundry
CODECOV_TOKEN: ci.datadog-agent.codecov_token
CWS_INSTRUMENTATION_BINARIES_DIR: bin/cws-instrumentation
DATADOG_AGENT_ARMBUILDIMAGES: v45979046-89a73a40
DATADOG_AGENT_ARMBUILDIMAGES_SUFFIX: ''
DATADOG_AGENT_BTF_GEN_BUILDIMAGES: v45979046-89a73a40
DATADOG_AGENT_BTF_GEN_BUILDIMAGES_SUFFIX: ''
DATADOG_AGENT_BUILDIMAGES: v45979046-89a73a40
DATADOG_AGENT_BUILDIMAGES_SUFFIX: ''
DATADOG_AGENT_EMBEDDED_PATH: /opt/datadog-agent/embedded
DATADOG_AGENT_SYSPROBE_BUILDIMAGES: v45979046-89a73a40
DATADOG_AGENT_SYSPROBE_BUILDIMAGES_SUFFIX: ''
DATADOG_AGENT_WINBUILDIMAGES: v45979046-89a73a40
DATADOG_AGENT_WINBUILDIMAGES_SUFFIX: ''
DD_AGENT_TESTING_DIR: $CI_PROJECT_DIR/test/kitchen
DD_PKG_VERSION: latest
DEB_GPG_KEY: ci.datadog-agent.deb_signing_private_key_${DEB_GPG_KEY_ID}
DEB_GPG_KEY_ID: c0962c7d
DEB_GPG_KEY_NAME: Datadog, Inc. APT key
DEB_RPM_TESTING_BUCKET_BRANCH: testing
DEB_S3_BUCKET: apt.datad0g.com
DEB_SIGNING_PASSPHRASE: ci.datadog-agent.deb_signing_key_passphrase_${DEB_GPG_KEY_ID}
DEB_TESTING_S3_BUCKET: apttesting.datad0g.com
DOCKER_REGISTRY_LOGIN: ci.datadog-agent.docker_hub_login
DOCKER_REGISTRY_PWD: ci.datadog-agent.docker_hub_pwd
DOCKER_REGISTRY_RO: dockerhub-readonly
DOCKER_REGISTRY_URL: docker.io
DOGSTATSD_BINARIES_DIR: bin/dogstatsd
+ E2E_PULUMI_CONFIG_PASSPHRASE: ci.datadog-agent.pulumi_password
E2E_TESTS_API_KEY: ci.datadog-agent.e2e_tests_api_key
E2E_TESTS_APP_KEY: ci.datadog-agent.e2e_tests_app_key
E2E_TESTS_AZURE_CLIENT_ID: ci.datadog-agent.e2e_tests_azure_client_id
E2E_TESTS_AZURE_CLIENT_SECRET: ci.datadog-agent.e2e_tests_azure_client_secret
E2E_TESTS_AZURE_SUBSCRIPTION_ID: ci.datadog-agent.e2e_tests_azure_subscription_id
E2E_TESTS_AZURE_TENANT_ID: ci.datadog-agent.e2e_tests_azure_tenant_id
E2E_TESTS_GCP_CREDENTIALS: ci.datadog-agent.e2e_tests_gcp_credentials
E2E_TESTS_RC_KEY: ci.datadog-agent.e2e_tests_rc_key
EXECUTOR_JOB_SECTION_ATTEMPTS: 2
FF_KUBERNETES_HONOR_ENTRYPOINT: true
FF_SCRIPT_SECTIONS: 1
GENERAL_ARTIFACTS_CACHE_BUCKET_URL: https://dd-agent-omnibus.s3.amazonaws.com
GET_SOURCES_ATTEMPTS: 2
GITHUB_PR_COMMENTER_APP_KEY: pr-commenter.github_app_key
GITHUB_PR_COMMENTER_INSTALLATION_ID: pr-commenter.github_installation_id
GITHUB_PR_COMMENTER_INTEGRATION_ID: pr-commenter.github_integration_id
GITLAB_FULL_API_TOKEN: ci.datadog-agent.gitlab_full_api_token
GITLAB_READ_API_TOKEN: ci.datadog-agent.gitlab_read_api_token
GITLAB_SCHEDULER_TOKEN: ci.datadog-agent.gitlab_pipelines_scheduler_token
GO_TEST_SKIP_FLAKE: 'true'
INSTALL_SCRIPT_API_KEY: ci.agent-linux-install-script.datadog_api_key_2
INTEGRATION_WHEELS_CACHE_BUCKET: dd-agent-omnibus
JIRA_READ_API_TOKEN: ci.datadog-agent.jira_read_api_token
KERNEL_MATRIX_TESTING_ARM_AMI_ID: ami-02fbd245475d6a63d
KERNEL_MATRIX_TESTING_X86_AMI_ID: ami-013ea43cbe85107c0
KITCHEN_AZURE_CLIENT_ID: ci.datadog-agent.azure_kitchen_client_id
KITCHEN_AZURE_CLIENT_SECRET: ci.datadog-agent.azure_kitchen_client_secret
KITCHEN_AZURE_SUBSCRIPTION_ID: ci.datadog-agent.azure_kitchen_subscription_id
KITCHEN_AZURE_TENANT_ID: ci.datadog-agent.azure_kitchen_tenant_id
KITCHEN_EC2_SSH_KEY: ci.datadog-agent.aws_ec2_kitchen_ssh_key
KITCHEN_INFRASTRUCTURE_FLAKES_RETRY: 2
MACOS_GITHUB_APP_1: macos-github-app-one
MACOS_GITHUB_APP_2: macos-github-app-two
MACOS_GITHUB_APP_ID: ci.datadog-agent.macos_github_app_id
MACOS_GITHUB_APP_ID_2: ci.datadog-agent.macos_github_app_id_2
MACOS_GITHUB_INSTALLATION_ID: ci.datadog-agent.macos_github_installation_id
MACOS_GITHUB_INSTALLATION_ID_2: ci.datadog-agent.macos_github_installation_id_2
MACOS_GITHUB_KEY: ci.datadog-agent.macos_github_key_b64
MACOS_GITHUB_KEY_2: ci.datadog-agent.macos_github_key_b64_2
MACOS_S3_BUCKET: dd-agent-macostesting
OMNIBUS_BASE_DIR: /omnibus
OMNIBUS_GIT_CACHE_DIR: /tmp/omnibus-git-cache
OMNIBUS_PACKAGE_DIR: $CI_PROJECT_DIR/omnibus/pkg/
OMNIBUS_PACKAGE_DIR_SUSE: $CI_PROJECT_DIR/omnibus/suse/pkg
PROCESS_S3_BUCKET: datad0g-process-agent
RELEASE_VERSION_6: nightly
RELEASE_VERSION_7: nightly-a7
RESTORE_CACHE_ATTEMPTS: 2
RPM_GPG_KEY: ci.datadog-agent.rpm_signing_private_key_${RPM_GPG_KEY_ID}
RPM_GPG_KEY_ID: b01082d3
RPM_GPG_KEY_NAME: Datadog, Inc. RPM key
RPM_S3_BUCKET: yum.datad0g.com
RPM_SIGNING_PASSPHRASE: ci.datadog-agent.rpm_signing_key_passphrase_${RPM_GPG_KEY_ID}
RPM_TESTING_S3_BUCKET: yumtesting.datad0g.com
RUN_E2E_TESTS: auto
RUN_KMT_TESTS: auto
RUN_UNIT_TESTS: auto
S3_ARTIFACTS_URI: s3://dd-ci-artefacts-build-stable/$CI_PROJECT_NAME/$CI_PIPELINE_ID
S3_CP_CMD: aws s3 cp $S3_CP_OPTIONS
S3_CP_OPTIONS: --no-progress --region us-east-1 --sse AES256
S3_DD_AGENT_OMNIBUS_BTFS_URI: s3://dd-agent-omnibus/btfs
S3_DD_AGENT_OMNIBUS_LLVM_URI: s3://dd-agent-omnibus/llvm
S3_DSD6_URI: s3://dsd6-staging
S3_OMNIBUS_CACHE_BUCKET: dd-ci-datadog-agent-omnibus-cache-build-stable
S3_PERMANENT_ARTIFACTS_URI: s3://dd-ci-persistent-artefacts-build-stable/$CI_PROJECT_NAME
S3_PROJECT_ARTIFACTS_URI: s3://dd-ci-artefacts-build-stable/$CI_PROJECT_NAME
S3_RELEASE_ARTIFACTS_URI: s3://dd-release-artifacts/$CI_PROJECT_NAME/$CI_PIPELINE_ID
S3_RELEASE_INSTALLER_ARTIFACTS_URI: s3://dd-release-artifacts/datadog-installer/$CI_PIPELINE_ID
S3_SBOM_STORAGE_URI: s3://sbom-root-us1-ddbuild-io/$CI_PROJECT_NAME/$CI_PIPELINE_ID
SLACK_AGENT_CI_TOKEN: ci.datadog-agent.slack_agent_ci_token
SMP_ACCOUNT_ID: ci.datadog-agent.single-machine-performance-account-id
SMP_AGENT_TEAM_ID: ci.datadog-agent.single-machine-performance-agent-team-id
SMP_API: ci.datadog-agent.single-machine-performance-api
SMP_BOT_ACCESS_KEY: ci.datadog-agent.single-machine-performance-bot-access-key
SMP_BOT_ACCESS_KEY_ID: ci.datadog-agent.single-machine-performance-bot-access-key-id
SSH_KEY: ci.datadog-agent.ssh_key
SSH_KEY_RSA: ci.datadog-agent.ssh_key_rsa
SSH_PUBLIC_KEY_RSA: ci.datadog-agent.ssh_public_key_rsa
STATIC_BINARIES_DIR: bin/static
SYSTEM_PROBE_BINARIES_DIR: bin/system-probe
USE_S3_CACHING: --omnibus-s3-cache
VCPKG_BLOB_SAS_URL: ci.datadog-agent-buildimages.vcpkg_blob_sas_url
WINDOWS_BUILDS_S3_BUCKET: $WIN_S3_BUCKET/builds
WINDOWS_POWERSHELL_DIR: $CI_PROJECT_DIR/signed_scripts
WINDOWS_TESTING_S3_BUCKET_A6: pipelines/A6/$CI_PIPELINE_ID
WINDOWS_TESTING_S3_BUCKET_A7: pipelines/A7/$CI_PIPELINE_ID
WINGET_PAT: ci.datadog-agent.winget_pat
WIN_S3_BUCKET: dd-agent-mstesting .on_npm_or_e2e_changes .on_npm_or_e2e_changes:
- if: $RUN_E2E_TESTS == "off"
when: never
- if: $CI_COMMIT_BRANCH =~ /^mq-working-branch-/
when: never
- if: $RUN_E2E_TESTS == "on"
when: on_success
- if: $CI_COMMIT_BRANCH == "main"
when: on_success
- if: $CI_COMMIT_BRANCH =~ /^[0-9]+\.[0-9]+\.x$/
when: on_success
- if: $CI_COMMIT_TAG =~ /^[0-9]+\.[0-9]+\.[0-9]+-rc\.[0-9]+$/
when: on_success
- changes:
compare_to: main
paths:
- test/new-e2e/pkg/**/*
- test/new-e2e/go.mod
- flakes.yaml
- changes:
compare_to: main
paths:
+ - pkg/network/**/*
- test/new-e2e/tests/npm/**/* new-e2e-containers new-e2e-containers:
after_script:
- $CI_PROJECT_DIR/tools/ci/junit_upload.sh
artifacts:
expire_in: 2 weeks
paths:
- $E2E_OUTPUT_DIR
- junit-*.tgz
reports:
annotations:
- $EXTERNAL_LINKS_PATH
when: always
before_script:
- mkdir -p $GOPATH/pkg/mod/cache && tar xJf modcache_e2e.tar.xz -C $GOPATH/pkg/mod/cache
- rm -f modcache_e2e.tar.xz
- mkdir -p ~/.aws
- $CI_PROJECT_DIR/tools/ci/fetch_secret.sh $AGENT_QA_PROFILE >> ~/.aws/config ||
exit $?
- export AWS_PROFILE=agent-qa-ci
- $CI_PROJECT_DIR/tools/ci/fetch_secret.sh $SSH_PUBLIC_KEY_RSA > $E2E_PUBLIC_KEY_PATH
|| exit $?
- touch $E2E_PRIVATE_KEY_PATH && chmod 600 $E2E_PRIVATE_KEY_PATH && $CI_PROJECT_DIR/tools/ci/fetch_secret.sh
$SSH_KEY_RSA > $E2E_PRIVATE_KEY_PATH || exit $?
- pulumi login "s3://dd-pulumi-state?region=us-east-1&awssdk=v2&profile=$AWS_PROFILE"
- ARM_CLIENT_ID=$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $E2E_TESTS_AZURE_CLIENT_ID)
|| exit $?; export ARM_CLIENT_ID
- ARM_CLIENT_SECRET=$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $E2E_TESTS_AZURE_CLIENT_SECRET)
|| exit $?; export ARM_CLIENT_SECRET
- ARM_TENANT_ID=$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $E2E_TESTS_AZURE_TENANT_ID)
|| exit $?; export ARM_TENANT_ID
- ARM_SUBSCRIPTION_ID=$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $E2E_TESTS_AZURE_SUBSCRIPTION_ID)
|| exit $?; export ARM_SUBSCRIPTION_ID
- GOOGLE_CREDENTIALS=$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $E2E_TESTS_GCP_CREDENTIALS)
|| exit $?; export GOOGLE_CREDENTIALS
- inv -e gitlab.generate-ci-visibility-links --output=$EXTERNAL_LINKS_PATH
image: 486234852809.dkr.ecr.us-east-1.amazonaws.com/ci/test-infra-definitions/runner$TEST_INFRA_DEFINITIONS_BUILDIMAGES_SUFFIX:$TEST_INFRA_DEFINITIONS_BUILDIMAGES
needs:
- go_e2e_deps
- qa_agent
- qa_dca
- qa_dogstatsd
parallel:
matrix:
- EXTRA_PARAMS: --run TestKindSuite -c ddinfra:kubernetesVersion=1.19
- EXTRA_PARAMS: --run TestKindSuite -c ddinfra:kubernetesVersion=1.22
- EXTRA_PARAMS: --run TestKindSuite -c ddinfra:kubernetesVersion=1.27
- EXTRA_PARAMS: --run TestKindSuite -c ddinfra:kubernetesVersion=1.29
- EXTRA_PARAMS: --run TestKindSuite -c ddinfra:osDescriptor=ubuntu:20.04
- EXTRA_PARAMS: --run TestKindSuite -c ddinfra:osDescriptor=ubuntu:22.04
- - EXTRA_PARAMS: --run TestEKSSuite
- EXTRA_PARAMS: --run TestECSSuite
- EXTRA_PARAMS: --run TestDockerSuite
- EXTRA_PARAMS: --skip "Test(Kind|EKS|ECS|Docker)Suite"
rules:
- if: $RUN_E2E_TESTS == "off"
when: never
- if: $CI_COMMIT_BRANCH =~ /^mq-working-branch-/
when: never
- if: $RUN_E2E_TESTS == "on"
when: on_success
- if: $CI_COMMIT_BRANCH == "main"
when: on_success
- if: $CI_COMMIT_BRANCH =~ /^[0-9]+\.[0-9]+\.x$/
when: on_success
- if: $CI_COMMIT_TAG =~ /^[0-9]+\.[0-9]+\.[0-9]+-rc\.[0-9]+$/
when: on_success
- changes:
compare_to: main
paths:
- test/new-e2e/pkg/**/*
- test/new-e2e/go.mod
- flakes.yaml
- changes:
compare_to: main
paths:
- comp/core/tagger/**/*
- comp/core/workloadmeta/**/*
- comp/core/autodiscovery/listeners/**/*
- comp/core/autodiscovery/providers/**/*
- comp/languagedetection/**/*
- pkg/clusteragent/admission/mutate/**/*
- pkg/collector/corechecks/cluster/**/*
- pkg/collector/corechecks/containers/**/*
- pkg/collector/corechecks/containerimage/**/*
- pkg/collector/corechecks/containerlifecycle/**/*
- pkg/collector/corechecks/sbom/**/*
- pkg/sbom/**/*
- pkg/util/clusteragent/**/*
- pkg/util/containerd/**/*
- pkg/util/containers/**/*
- pkg/util/docker/**/*
- pkg/util/ecs/**/*
- pkg/util/kubernetes/**/*
- pkg/util/cgroups/**/*
- pkg/util/trivy/**/*
- test/new-e2e/tests/containers/**/*
- test/new-e2e/go.mod
when: on_success
- if: $CI_COMMIT_BRANCH =~ /^mq-working-branch-/
when: never
- allow_failure: true
when: manual
script:
- inv -e new-e2e-tests.run --targets $TARGETS -c ddagent:imagePullRegistry=669783387624.dkr.ecr.us-east-1.amazonaws.com
-c ddagent:imagePullUsername=AWS -c ddagent:imagePullPassword=$(aws ecr get-login-password)
--junit-tar junit-${CI_JOB_ID}.tgz ${EXTRA_PARAMS} --test-washer
stage: e2e
tags:
- arch:amd64
variables:
E2E_COMMIT_SHA: $CI_COMMIT_SHORT_SHA
E2E_KEY_PAIR_NAME: datadog-agent-ci-rsa
E2E_OUTPUT_DIR: $CI_PROJECT_DIR/e2e-output
E2E_PIPELINE_ID: $CI_PIPELINE_ID
E2E_PRIVATE_KEY_PATH: /tmp/agent-qa-ssh-key
E2E_PUBLIC_KEY_PATH: /tmp/agent-qa-ssh-key.pub
EXTERNAL_LINKS_PATH: external_links_$CI_JOB_ID.json
KUBERNETES_CPU_REQUEST: 6
KUBERNETES_MEMORY_LIMIT: 16Gi
KUBERNETES_MEMORY_REQUEST: 12Gi
SHOULD_RUN_IN_FLAKES_FINDER: 'true'
TARGETS: ./tests/containers
TEAM: container-integrations new-e2e-npm-docker new-e2e-npm-docker:
after_script:
- $CI_PROJECT_DIR/tools/ci/junit_upload.sh
artifacts:
expire_in: 2 weeks
paths:
- $E2E_OUTPUT_DIR
- junit-*.tgz
reports:
annotations:
- $EXTERNAL_LINKS_PATH
when: always
before_script:
- mkdir -p $GOPATH/pkg/mod/cache && tar xJf modcache_e2e.tar.xz -C $GOPATH/pkg/mod/cache
- rm -f modcache_e2e.tar.xz
- mkdir -p ~/.aws
- $CI_PROJECT_DIR/tools/ci/fetch_secret.sh $AGENT_QA_PROFILE >> ~/.aws/config ||
exit $?
- export AWS_PROFILE=agent-qa-ci
- $CI_PROJECT_DIR/tools/ci/fetch_secret.sh $SSH_PUBLIC_KEY_RSA > $E2E_PUBLIC_KEY_PATH
|| exit $?
- touch $E2E_PRIVATE_KEY_PATH && chmod 600 $E2E_PRIVATE_KEY_PATH && $CI_PROJECT_DIR/tools/ci/fetch_secret.sh
$SSH_KEY_RSA > $E2E_PRIVATE_KEY_PATH || exit $?
- pulumi login "s3://dd-pulumi-state?region=us-east-1&awssdk=v2&profile=$AWS_PROFILE"
- ARM_CLIENT_ID=$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $E2E_TESTS_AZURE_CLIENT_ID)
|| exit $?; export ARM_CLIENT_ID
- ARM_CLIENT_SECRET=$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $E2E_TESTS_AZURE_CLIENT_SECRET)
|| exit $?; export ARM_CLIENT_SECRET
- ARM_TENANT_ID=$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $E2E_TESTS_AZURE_TENANT_ID)
|| exit $?; export ARM_TENANT_ID
- ARM_SUBSCRIPTION_ID=$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $E2E_TESTS_AZURE_SUBSCRIPTION_ID)
|| exit $?; export ARM_SUBSCRIPTION_ID
- GOOGLE_CREDENTIALS=$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $E2E_TESTS_GCP_CREDENTIALS)
|| exit $?; export GOOGLE_CREDENTIALS
- inv -e gitlab.generate-ci-visibility-links --output=$EXTERNAL_LINKS_PATH
image: 486234852809.dkr.ecr.us-east-1.amazonaws.com/ci/test-infra-definitions/runner$TEST_INFRA_DEFINITIONS_BUILDIMAGES_SUFFIX:$TEST_INFRA_DEFINITIONS_BUILDIMAGES
needs:
- go_e2e_deps
- qa_dca
- qa_agent
rules:
- if: $RUN_E2E_TESTS == "off"
when: never
- if: $CI_COMMIT_BRANCH =~ /^mq-working-branch-/
when: never
- if: $RUN_E2E_TESTS == "on"
when: on_success
- if: $CI_COMMIT_BRANCH == "main"
when: on_success
- if: $CI_COMMIT_BRANCH =~ /^[0-9]+\.[0-9]+\.x$/
when: on_success
- if: $CI_COMMIT_TAG =~ /^[0-9]+\.[0-9]+\.[0-9]+-rc\.[0-9]+$/
when: on_success
- changes:
compare_to: main
paths:
- test/new-e2e/pkg/**/*
- test/new-e2e/go.mod
- flakes.yaml
- changes:
compare_to: main
paths:
+ - pkg/network/**/*
- test/new-e2e/tests/npm/**/*
- if: $CI_COMMIT_BRANCH =~ /^mq-working-branch-/
when: never
- allow_failure: true
when: manual
script:
- inv -e new-e2e-tests.run --targets $TARGETS -c ddagent:imagePullRegistry=669783387624.dkr.ecr.us-east-1.amazonaws.com
-c ddagent:imagePullUsername=AWS -c ddagent:imagePullPassword=$(aws ecr get-login-password)
--junit-tar junit-${CI_JOB_ID}.tgz ${EXTRA_PARAMS} --test-washer
stage: e2e
tags:
- arch:amd64
variables:
E2E_COMMIT_SHA: $CI_COMMIT_SHORT_SHA
E2E_KEY_PAIR_NAME: datadog-agent-ci-rsa
E2E_OUTPUT_DIR: $CI_PROJECT_DIR/e2e-output
E2E_PIPELINE_ID: $CI_PIPELINE_ID
E2E_PRIVATE_KEY_PATH: /tmp/agent-qa-ssh-key
E2E_PUBLIC_KEY_PATH: /tmp/agent-qa-ssh-key.pub
EXTERNAL_LINKS_PATH: external_links_$CI_JOB_ID.json
- EXTRA_PARAMS: --run "Test(ECSVM|EC2VMContainerized|EKSVM)Suite"
? ------
+ EXTRA_PARAMS: --run "Test(ECSVM|EC2VMContainerized)Suite"
KUBERNETES_CPU_REQUEST: 6
KUBERNETES_MEMORY_LIMIT: 16Gi
KUBERNETES_MEMORY_REQUEST: 12Gi
SHOULD_RUN_IN_FLAKES_FINDER: 'true'
TARGETS: ./tests/npm
TEAM: network-performance-monitoring new-e2e-npm-packages new-e2e-npm-packages:
after_script:
- $CI_PROJECT_DIR/tools/ci/junit_upload.sh
artifacts:
expire_in: 2 weeks
paths:
- $E2E_OUTPUT_DIR
- junit-*.tgz
reports:
annotations:
- $EXTERNAL_LINKS_PATH
when: always
before_script:
- mkdir -p $GOPATH/pkg/mod/cache && tar xJf modcache_e2e.tar.xz -C $GOPATH/pkg/mod/cache
- rm -f modcache_e2e.tar.xz
- mkdir -p ~/.aws
- $CI_PROJECT_DIR/tools/ci/fetch_secret.sh $AGENT_QA_PROFILE >> ~/.aws/config ||
exit $?
- export AWS_PROFILE=agent-qa-ci
- $CI_PROJECT_DIR/tools/ci/fetch_secret.sh $SSH_PUBLIC_KEY_RSA > $E2E_PUBLIC_KEY_PATH
|| exit $?
- touch $E2E_PRIVATE_KEY_PATH && chmod 600 $E2E_PRIVATE_KEY_PATH && $CI_PROJECT_DIR/tools/ci/fetch_secret.sh
$SSH_KEY_RSA > $E2E_PRIVATE_KEY_PATH || exit $?
- pulumi login "s3://dd-pulumi-state?region=us-east-1&awssdk=v2&profile=$AWS_PROFILE"
- ARM_CLIENT_ID=$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $E2E_TESTS_AZURE_CLIENT_ID)
|| exit $?; export ARM_CLIENT_ID
- ARM_CLIENT_SECRET=$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $E2E_TESTS_AZURE_CLIENT_SECRET)
|| exit $?; export ARM_CLIENT_SECRET
- ARM_TENANT_ID=$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $E2E_TESTS_AZURE_TENANT_ID)
|| exit $?; export ARM_TENANT_ID
- ARM_SUBSCRIPTION_ID=$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $E2E_TESTS_AZURE_SUBSCRIPTION_ID)
|| exit $?; export ARM_SUBSCRIPTION_ID
- GOOGLE_CREDENTIALS=$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $E2E_TESTS_GCP_CREDENTIALS)
|| exit $?; export GOOGLE_CREDENTIALS
- inv -e gitlab.generate-ci-visibility-links --output=$EXTERNAL_LINKS_PATH
image: 486234852809.dkr.ecr.us-east-1.amazonaws.com/ci/test-infra-definitions/runner$TEST_INFRA_DEFINITIONS_BUILDIMAGES_SUFFIX:$TEST_INFRA_DEFINITIONS_BUILDIMAGES
needs:
- go_e2e_deps
- deploy_deb_testing-a7_x64
- deploy_rpm_testing-a7_x64
- deploy_windows_testing-a7
rules:
- if: $RUN_E2E_TESTS == "off"
when: never
- if: $CI_COMMIT_BRANCH =~ /^mq-working-branch-/
when: never
- if: $RUN_E2E_TESTS == "on"
when: on_success
- if: $CI_COMMIT_BRANCH == "main"
when: on_success
- if: $CI_COMMIT_BRANCH =~ /^[0-9]+\.[0-9]+\.x$/
when: on_success
- if: $CI_COMMIT_TAG =~ /^[0-9]+\.[0-9]+\.[0-9]+-rc\.[0-9]+$/
when: on_success
- changes:
compare_to: main
paths:
- test/new-e2e/pkg/**/*
- test/new-e2e/go.mod
- flakes.yaml
- changes:
compare_to: main
paths:
+ - pkg/network/**/*
- test/new-e2e/tests/npm/**/*
- if: $CI_COMMIT_BRANCH =~ /^mq-working-branch-/
when: never
- allow_failure: true
when: manual
script:
- inv -e new-e2e-tests.run --targets $TARGETS -c ddagent:imagePullRegistry=669783387624.dkr.ecr.us-east-1.amazonaws.com
-c ddagent:imagePullUsername=AWS -c ddagent:imagePullPassword=$(aws ecr get-login-password)
--junit-tar junit-${CI_JOB_ID}.tgz ${EXTRA_PARAMS} --test-washer
stage: e2e
tags:
- arch:amd64
variables:
E2E_COMMIT_SHA: $CI_COMMIT_SHORT_SHA
E2E_KEY_PAIR_NAME: datadog-agent-ci-rsa
E2E_OUTPUT_DIR: $CI_PROJECT_DIR/e2e-output
E2E_PIPELINE_ID: $CI_PIPELINE_ID
E2E_PRIVATE_KEY_PATH: /tmp/agent-qa-ssh-key
E2E_PUBLIC_KEY_PATH: /tmp/agent-qa-ssh-key.pub
EXTERNAL_LINKS_PATH: external_links_$CI_JOB_ID.json
EXTRA_PARAMS: --run "TestEC2(VM|VMSELinux|VMWKit)Suite"
KUBERNETES_CPU_REQUEST: 6
KUBERNETES_MEMORY_LIMIT: 16Gi
KUBERNETES_MEMORY_REQUEST: 12Gi
SHOULD_RUN_IN_FLAKES_FINDER: 'true'
TARGETS: ./tests/npm
TEAM: network-performance-monitoring Added Jobsnew-e2e-containers-eksnew-e2e-containers-eks:
after_script:
- $CI_PROJECT_DIR/tools/ci/junit_upload.sh
artifacts:
expire_in: 2 weeks
paths:
- $E2E_OUTPUT_DIR
- junit-*.tgz
reports:
annotations:
- $EXTERNAL_LINKS_PATH
when: always
before_script:
- mkdir -p $GOPATH/pkg/mod/cache && tar xJf modcache_e2e.tar.xz -C $GOPATH/pkg/mod/cache
- rm -f modcache_e2e.tar.xz
- mkdir -p ~/.aws
- $CI_PROJECT_DIR/tools/ci/fetch_secret.sh $AGENT_QA_PROFILE >> ~/.aws/config ||
exit $?
- export AWS_PROFILE=agent-qa-ci
- $CI_PROJECT_DIR/tools/ci/fetch_secret.sh $SSH_PUBLIC_KEY_RSA > $E2E_PUBLIC_KEY_PATH
|| exit $?
- touch $E2E_PRIVATE_KEY_PATH && chmod 600 $E2E_PRIVATE_KEY_PATH && $CI_PROJECT_DIR/tools/ci/fetch_secret.sh
$SSH_KEY_RSA > $E2E_PRIVATE_KEY_PATH || exit $?
- pulumi login "s3://dd-pulumi-state?region=us-east-1&awssdk=v2&profile=$AWS_PROFILE"
- ARM_CLIENT_ID=$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $E2E_TESTS_AZURE_CLIENT_ID)
|| exit $?; export ARM_CLIENT_ID
- ARM_CLIENT_SECRET=$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $E2E_TESTS_AZURE_CLIENT_SECRET)
|| exit $?; export ARM_CLIENT_SECRET
- ARM_TENANT_ID=$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $E2E_TESTS_AZURE_TENANT_ID)
|| exit $?; export ARM_TENANT_ID
- ARM_SUBSCRIPTION_ID=$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $E2E_TESTS_AZURE_SUBSCRIPTION_ID)
|| exit $?; export ARM_SUBSCRIPTION_ID
- GOOGLE_CREDENTIALS=$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $E2E_TESTS_GCP_CREDENTIALS)
|| exit $?; export GOOGLE_CREDENTIALS
- inv -e gitlab.generate-ci-visibility-links --output=$EXTERNAL_LINKS_PATH
image: 486234852809.dkr.ecr.us-east-1.amazonaws.com/ci/test-infra-definitions/runner$TEST_INFRA_DEFINITIONS_BUILDIMAGES_SUFFIX:$TEST_INFRA_DEFINITIONS_BUILDIMAGES
needs:
- go_e2e_deps
- new-e2e-containers-eks-init
- qa_agent
- qa_dca
rules:
- if: $RUN_E2E_TESTS == "off"
when: never
- if: $CI_COMMIT_BRANCH =~ /^mq-working-branch-/
when: never
- if: $RUN_E2E_TESTS == "on"
when: on_success
- if: $CI_COMMIT_BRANCH == "main"
when: on_success
- if: $CI_COMMIT_BRANCH =~ /^[0-9]+\.[0-9]+\.x$/
when: on_success
- if: $CI_COMMIT_TAG =~ /^[0-9]+\.[0-9]+\.[0-9]+-rc\.[0-9]+$/
when: on_success
- changes:
compare_to: main
paths:
- test/new-e2e/pkg/**/*
- test/new-e2e/go.mod
- flakes.yaml
- changes:
compare_to: main
paths:
- comp/core/tagger/**/*
- comp/core/workloadmeta/**/*
- comp/core/autodiscovery/listeners/**/*
- comp/core/autodiscovery/providers/**/*
- comp/languagedetection/**/*
- pkg/clusteragent/admission/mutate/**/*
- pkg/collector/corechecks/cluster/**/*
- pkg/collector/corechecks/containers/**/*
- pkg/collector/corechecks/containerimage/**/*
- pkg/collector/corechecks/containerlifecycle/**/*
- pkg/collector/corechecks/sbom/**/*
- pkg/sbom/**/*
- pkg/util/clusteragent/**/*
- pkg/util/containerd/**/*
- pkg/util/containers/**/*
- pkg/util/docker/**/*
- pkg/util/ecs/**/*
- pkg/util/kubernetes/**/*
- pkg/util/cgroups/**/*
- pkg/util/trivy/**/*
- test/new-e2e/tests/containers/**/*
- test/new-e2e/go.mod
when: on_success
- if: $CI_COMMIT_BRANCH =~ /^mq-working-branch-/
when: never
- allow_failure: true
when: manual
script:
- inv -e new-e2e-tests.run --targets $TARGETS -c ddagent:imagePullRegistry=669783387624.dkr.ecr.us-east-1.amazonaws.com
-c ddagent:imagePullUsername=AWS -c ddagent:imagePullPassword=$(aws ecr get-login-password)
--junit-tar junit-${CI_JOB_ID}.tgz ${EXTRA_PARAMS} --test-washer
stage: e2e
tags:
- arch:amd64
variables:
E2E_COMMIT_SHA: $CI_COMMIT_SHORT_SHA
E2E_KEY_PAIR_NAME: datadog-agent-ci-rsa
E2E_OUTPUT_DIR: $CI_PROJECT_DIR/e2e-output
E2E_PIPELINE_ID: $CI_PIPELINE_ID
E2E_PRE_INITIALIZED: 'true'
E2E_PRIVATE_KEY_PATH: /tmp/agent-qa-ssh-key
E2E_PUBLIC_KEY_PATH: /tmp/agent-qa-ssh-key.pub
EXTERNAL_LINKS_PATH: external_links_$CI_JOB_ID.json
EXTRA_PARAMS: --run TestEKSSuite
KUBERNETES_CPU_REQUEST: 6
KUBERNETES_MEMORY_LIMIT: 16Gi
KUBERNETES_MEMORY_REQUEST: 12Gi
SHOULD_RUN_IN_FLAKES_FINDER: 'true'
TARGETS: ./tests/containers
TEAM: container-integrations new-e2e-containers-eks-initnew-e2e-containers-eks-init:
after_script:
- $CI_PROJECT_DIR/tools/ci/junit_upload.sh
allow_failure: true
artifacts:
expire_in: 2 weeks
paths:
- $E2E_OUTPUT_DIR
- junit-*.tgz
reports:
annotations:
- $EXTERNAL_LINKS_PATH
when: always
before_script:
- mkdir -p $GOPATH/pkg/mod/cache && tar xJf modcache_e2e.tar.xz -C $GOPATH/pkg/mod/cache
- rm -f modcache_e2e.tar.xz
- mkdir -p ~/.aws
- $CI_PROJECT_DIR/tools/ci/fetch_secret.sh $AGENT_QA_PROFILE >> ~/.aws/config ||
exit $?
- export AWS_PROFILE=agent-qa-ci
- $CI_PROJECT_DIR/tools/ci/fetch_secret.sh $SSH_PUBLIC_KEY_RSA > $E2E_PUBLIC_KEY_PATH
|| exit $?
- touch $E2E_PRIVATE_KEY_PATH && chmod 600 $E2E_PRIVATE_KEY_PATH && $CI_PROJECT_DIR/tools/ci/fetch_secret.sh
$SSH_KEY_RSA > $E2E_PRIVATE_KEY_PATH || exit $?
- pulumi login "s3://dd-pulumi-state?region=us-east-1&awssdk=v2&profile=$AWS_PROFILE"
- ARM_CLIENT_ID=$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $E2E_TESTS_AZURE_CLIENT_ID)
|| exit $?; export ARM_CLIENT_ID
- ARM_CLIENT_SECRET=$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $E2E_TESTS_AZURE_CLIENT_SECRET)
|| exit $?; export ARM_CLIENT_SECRET
- ARM_TENANT_ID=$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $E2E_TESTS_AZURE_TENANT_ID)
|| exit $?; export ARM_TENANT_ID
- ARM_SUBSCRIPTION_ID=$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $E2E_TESTS_AZURE_SUBSCRIPTION_ID)
|| exit $?; export ARM_SUBSCRIPTION_ID
- GOOGLE_CREDENTIALS=$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $E2E_TESTS_GCP_CREDENTIALS)
|| exit $?; export GOOGLE_CREDENTIALS
- inv -e gitlab.generate-ci-visibility-links --output=$EXTERNAL_LINKS_PATH
image: 486234852809.dkr.ecr.us-east-1.amazonaws.com/ci/test-infra-definitions/runner$TEST_INFRA_DEFINITIONS_BUILDIMAGES_SUFFIX:$TEST_INFRA_DEFINITIONS_BUILDIMAGES
needs:
- go_e2e_deps
rules:
- if: $RUN_E2E_TESTS == "off"
when: never
- if: $CI_COMMIT_BRANCH =~ /^mq-working-branch-/
when: never
- if: $RUN_E2E_TESTS == "on"
when: on_success
- if: $CI_COMMIT_BRANCH == "main"
when: on_success
- if: $CI_COMMIT_BRANCH =~ /^[0-9]+\.[0-9]+\.x$/
when: on_success
- if: $CI_COMMIT_TAG =~ /^[0-9]+\.[0-9]+\.[0-9]+-rc\.[0-9]+$/
when: on_success
- changes:
compare_to: main
paths:
- test/new-e2e/pkg/**/*
- test/new-e2e/go.mod
- flakes.yaml
- changes:
compare_to: main
paths:
- comp/core/tagger/**/*
- comp/core/workloadmeta/**/*
- comp/core/autodiscovery/listeners/**/*
- comp/core/autodiscovery/providers/**/*
- comp/languagedetection/**/*
- pkg/clusteragent/admission/mutate/**/*
- pkg/collector/corechecks/cluster/**/*
- pkg/collector/corechecks/containers/**/*
- pkg/collector/corechecks/containerimage/**/*
- pkg/collector/corechecks/containerlifecycle/**/*
- pkg/collector/corechecks/sbom/**/*
- pkg/sbom/**/*
- pkg/util/clusteragent/**/*
- pkg/util/containerd/**/*
- pkg/util/containers/**/*
- pkg/util/docker/**/*
- pkg/util/ecs/**/*
- pkg/util/kubernetes/**/*
- pkg/util/cgroups/**/*
- pkg/util/trivy/**/*
- test/new-e2e/tests/containers/**/*
- test/new-e2e/go.mod
when: on_success
- if: $CI_COMMIT_BRANCH =~ /^mq-working-branch-/
when: never
- allow_failure: true
when: manual
script:
- inv -e new-e2e-tests.run --targets $TARGETS -c ddagent:imagePullRegistry=669783387624.dkr.ecr.us-east-1.amazonaws.com
-c ddagent:imagePullUsername=AWS -c ddagent:imagePullPassword=$(aws ecr get-login-password)
--junit-tar junit-${CI_JOB_ID}.tgz ${EXTRA_PARAMS} --test-washer
stage: e2e_init
tags:
- arch:amd64
variables:
E2E_COMMIT_SHA: $CI_COMMIT_SHORT_SHA
E2E_INIT_ONLY: 'true'
E2E_KEY_PAIR_NAME: datadog-agent-ci-rsa
E2E_OUTPUT_DIR: $CI_PROJECT_DIR/e2e-output
E2E_PIPELINE_ID: $CI_PIPELINE_ID
E2E_PRIVATE_KEY_PATH: /tmp/agent-qa-ssh-key
E2E_PUBLIC_KEY_PATH: /tmp/agent-qa-ssh-key.pub
EXTERNAL_LINKS_PATH: external_links_$CI_JOB_ID.json
EXTRA_PARAMS: --run TestEKSSuite
KUBERNETES_CPU_REQUEST: 6
KUBERNETES_MEMORY_LIMIT: 16Gi
KUBERNETES_MEMORY_REQUEST: 12Gi
SHOULD_RUN_IN_FLAKES_FINDER: 'true'
TARGETS: ./tests/containers
TEAM: container-integrations new-e2e-eks-cleanup-on-failurenew-e2e-eks-cleanup-on-failure:
allow_failure: true
dependencies:
- go_e2e_deps
image: 486234852809.dkr.ecr.us-east-1.amazonaws.com/ci/test-infra-definitions/runner$TEST_INFRA_DEFINITIONS_BUILDIMAGES_SUFFIX:$TEST_INFRA_DEFINITIONS_BUILDIMAGES
script:
- mkdir -p ~/.aws
- $CI_PROJECT_DIR/tools/ci/fetch_secret.sh $AGENT_QA_PROFILE >> ~/.aws/config ||
exit $?
- export AWS_PROFILE=agent-qa-ci
- $CI_PROJECT_DIR/tools/ci/fetch_secret.sh $SSH_PUBLIC_KEY_RSA > $E2E_PUBLIC_KEY_PATH
|| exit $?
- touch $E2E_PRIVATE_KEY_PATH && chmod 600 $E2E_PRIVATE_KEY_PATH && $CI_PROJECT_DIR/tools/ci/fetch_secret.sh
$SSH_KEY_RSA > $E2E_PRIVATE_KEY_PATH || exit $?
- PULUMI_CONFIG_PASSPHRASE=$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $E2E_PULUMI_CONFIG_PASSPHRASE)
|| exit $?; export PULUMI_CONFIG_PASSPHRASE
- pulumi login "s3://dd-pulumi-state?region=us-east-1&awssdk=v2&profile=$AWS_PROFILE"
- inv -e new-e2e-tests.cleanup-remote-stacks --stack-regex "ci-$CI_PIPELINE_ID.*eks.*"
--pulumi-backend=dd-pulumi-state
stage: e2e_cleanup
tags:
- arch:amd64
variables:
E2E_PIPELINE_ID: $CI_PIPELINE_ID
E2E_PRIVATE_KEY_PATH: /tmp/agent-qa-ssh-key
E2E_PUBLIC_KEY_PATH: /tmp/agent-qa-ssh-key.pub
when: always new-e2e-npm-eksnew-e2e-npm-eks:
after_script:
- $CI_PROJECT_DIR/tools/ci/junit_upload.sh
artifacts:
expire_in: 2 weeks
paths:
- $E2E_OUTPUT_DIR
- junit-*.tgz
reports:
annotations:
- $EXTERNAL_LINKS_PATH
when: always
before_script:
- mkdir -p $GOPATH/pkg/mod/cache && tar xJf modcache_e2e.tar.xz -C $GOPATH/pkg/mod/cache
- rm -f modcache_e2e.tar.xz
- mkdir -p ~/.aws
- $CI_PROJECT_DIR/tools/ci/fetch_secret.sh $AGENT_QA_PROFILE >> ~/.aws/config ||
exit $?
- export AWS_PROFILE=agent-qa-ci
- $CI_PROJECT_DIR/tools/ci/fetch_secret.sh $SSH_PUBLIC_KEY_RSA > $E2E_PUBLIC_KEY_PATH
|| exit $?
- touch $E2E_PRIVATE_KEY_PATH && chmod 600 $E2E_PRIVATE_KEY_PATH && $CI_PROJECT_DIR/tools/ci/fetch_secret.sh
$SSH_KEY_RSA > $E2E_PRIVATE_KEY_PATH || exit $?
- pulumi login "s3://dd-pulumi-state?region=us-east-1&awssdk=v2&profile=$AWS_PROFILE"
- ARM_CLIENT_ID=$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $E2E_TESTS_AZURE_CLIENT_ID)
|| exit $?; export ARM_CLIENT_ID
- ARM_CLIENT_SECRET=$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $E2E_TESTS_AZURE_CLIENT_SECRET)
|| exit $?; export ARM_CLIENT_SECRET
- ARM_TENANT_ID=$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $E2E_TESTS_AZURE_TENANT_ID)
|| exit $?; export ARM_TENANT_ID
- ARM_SUBSCRIPTION_ID=$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $E2E_TESTS_AZURE_SUBSCRIPTION_ID)
|| exit $?; export ARM_SUBSCRIPTION_ID
- GOOGLE_CREDENTIALS=$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $E2E_TESTS_GCP_CREDENTIALS)
|| exit $?; export GOOGLE_CREDENTIALS
- inv -e gitlab.generate-ci-visibility-links --output=$EXTERNAL_LINKS_PATH
image: 486234852809.dkr.ecr.us-east-1.amazonaws.com/ci/test-infra-definitions/runner$TEST_INFRA_DEFINITIONS_BUILDIMAGES_SUFFIX:$TEST_INFRA_DEFINITIONS_BUILDIMAGES
needs:
- go_e2e_deps
- new-e2e-npm-eks-init
- qa_agent
- qa_dca
rules:
- if: $RUN_E2E_TESTS == "off"
when: never
- if: $CI_COMMIT_BRANCH =~ /^mq-working-branch-/
when: never
- if: $RUN_E2E_TESTS == "on"
when: on_success
- if: $CI_COMMIT_BRANCH == "main"
when: on_success
- if: $CI_COMMIT_BRANCH =~ /^[0-9]+\.[0-9]+\.x$/
when: on_success
- if: $CI_COMMIT_TAG =~ /^[0-9]+\.[0-9]+\.[0-9]+-rc\.[0-9]+$/
when: on_success
- changes:
compare_to: main
paths:
- test/new-e2e/pkg/**/*
- test/new-e2e/go.mod
- flakes.yaml
- changes:
compare_to: main
paths:
- pkg/network/**/*
- test/new-e2e/tests/npm/**/*
- if: $CI_COMMIT_BRANCH =~ /^mq-working-branch-/
when: never
- allow_failure: true
when: manual
script:
- inv -e new-e2e-tests.run --targets $TARGETS -c ddagent:imagePullRegistry=669783387624.dkr.ecr.us-east-1.amazonaws.com
-c ddagent:imagePullUsername=AWS -c ddagent:imagePullPassword=$(aws ecr get-login-password)
--junit-tar junit-${CI_JOB_ID}.tgz ${EXTRA_PARAMS} --test-washer
stage: e2e
tags:
- arch:amd64
variables:
E2E_COMMIT_SHA: $CI_COMMIT_SHORT_SHA
E2E_KEY_PAIR_NAME: datadog-agent-ci-rsa
E2E_OUTPUT_DIR: $CI_PROJECT_DIR/e2e-output
E2E_PIPELINE_ID: $CI_PIPELINE_ID
E2E_PRE_INITIALIZED: 'true'
E2E_PRIVATE_KEY_PATH: /tmp/agent-qa-ssh-key
E2E_PUBLIC_KEY_PATH: /tmp/agent-qa-ssh-key.pub
EXTERNAL_LINKS_PATH: external_links_$CI_JOB_ID.json
EXTRA_PARAMS: --run "TestEKSVMSuite"
KUBERNETES_CPU_REQUEST: 6
KUBERNETES_MEMORY_LIMIT: 16Gi
KUBERNETES_MEMORY_REQUEST: 12Gi
SHOULD_RUN_IN_FLAKES_FINDER: 'true'
TARGETS: ./tests/npm
TEAM: network-performance-monitoring new-e2e-npm-eks-initnew-e2e-npm-eks-init:
after_script:
- $CI_PROJECT_DIR/tools/ci/junit_upload.sh
allow_failure: true
artifacts:
expire_in: 2 weeks
paths:
- $E2E_OUTPUT_DIR
- junit-*.tgz
reports:
annotations:
- $EXTERNAL_LINKS_PATH
when: always
before_script:
- mkdir -p $GOPATH/pkg/mod/cache && tar xJf modcache_e2e.tar.xz -C $GOPATH/pkg/mod/cache
- rm -f modcache_e2e.tar.xz
- mkdir -p ~/.aws
- $CI_PROJECT_DIR/tools/ci/fetch_secret.sh $AGENT_QA_PROFILE >> ~/.aws/config ||
exit $?
- export AWS_PROFILE=agent-qa-ci
- $CI_PROJECT_DIR/tools/ci/fetch_secret.sh $SSH_PUBLIC_KEY_RSA > $E2E_PUBLIC_KEY_PATH
|| exit $?
- touch $E2E_PRIVATE_KEY_PATH && chmod 600 $E2E_PRIVATE_KEY_PATH && $CI_PROJECT_DIR/tools/ci/fetch_secret.sh
$SSH_KEY_RSA > $E2E_PRIVATE_KEY_PATH || exit $?
- pulumi login "s3://dd-pulumi-state?region=us-east-1&awssdk=v2&profile=$AWS_PROFILE"
- ARM_CLIENT_ID=$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $E2E_TESTS_AZURE_CLIENT_ID)
|| exit $?; export ARM_CLIENT_ID
- ARM_CLIENT_SECRET=$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $E2E_TESTS_AZURE_CLIENT_SECRET)
|| exit $?; export ARM_CLIENT_SECRET
- ARM_TENANT_ID=$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $E2E_TESTS_AZURE_TENANT_ID)
|| exit $?; export ARM_TENANT_ID
- ARM_SUBSCRIPTION_ID=$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $E2E_TESTS_AZURE_SUBSCRIPTION_ID)
|| exit $?; export ARM_SUBSCRIPTION_ID
- GOOGLE_CREDENTIALS=$($CI_PROJECT_DIR/tools/ci/fetch_secret.sh $E2E_TESTS_GCP_CREDENTIALS)
|| exit $?; export GOOGLE_CREDENTIALS
- inv -e gitlab.generate-ci-visibility-links --output=$EXTERNAL_LINKS_PATH
image: 486234852809.dkr.ecr.us-east-1.amazonaws.com/ci/test-infra-definitions/runner$TEST_INFRA_DEFINITIONS_BUILDIMAGES_SUFFIX:$TEST_INFRA_DEFINITIONS_BUILDIMAGES
needs:
- go_e2e_deps
rules:
- if: $RUN_E2E_TESTS == "off"
when: never
- if: $CI_COMMIT_BRANCH =~ /^mq-working-branch-/
when: never
- if: $RUN_E2E_TESTS == "on"
when: on_success
- if: $CI_COMMIT_BRANCH == "main"
when: on_success
- if: $CI_COMMIT_BRANCH =~ /^[0-9]+\.[0-9]+\.x$/
when: on_success
- if: $CI_COMMIT_TAG =~ /^[0-9]+\.[0-9]+\.[0-9]+-rc\.[0-9]+$/
when: on_success
- changes:
compare_to: main
paths:
- test/new-e2e/pkg/**/*
- test/new-e2e/go.mod
- flakes.yaml
- changes:
compare_to: main
paths:
- pkg/network/**/*
- test/new-e2e/tests/npm/**/*
- if: $CI_COMMIT_BRANCH =~ /^mq-working-branch-/
when: never
- allow_failure: true
when: manual
script:
- inv -e new-e2e-tests.run --targets $TARGETS -c ddagent:imagePullRegistry=669783387624.dkr.ecr.us-east-1.amazonaws.com
-c ddagent:imagePullUsername=AWS -c ddagent:imagePullPassword=$(aws ecr get-login-password)
--junit-tar junit-${CI_JOB_ID}.tgz ${EXTRA_PARAMS} --test-washer
stage: e2e_init
tags:
- arch:amd64
variables:
E2E_COMMIT_SHA: $CI_COMMIT_SHORT_SHA
E2E_INIT_ONLY: 'true'
E2E_KEY_PAIR_NAME: datadog-agent-ci-rsa
E2E_OUTPUT_DIR: $CI_PROJECT_DIR/e2e-output
E2E_PIPELINE_ID: $CI_PIPELINE_ID
E2E_PRIVATE_KEY_PATH: /tmp/agent-qa-ssh-key
E2E_PUBLIC_KEY_PATH: /tmp/agent-qa-ssh-key.pub
EXTERNAL_LINKS_PATH: external_links_$CI_JOB_ID.json
EXTRA_PARAMS: --run "TestEKSVMSuite"
KUBERNETES_CPU_REQUEST: 6
KUBERNETES_MEMORY_LIMIT: 16Gi
KUBERNETES_MEMORY_REQUEST: 12Gi
SHOULD_RUN_IN_FLAKES_FINDER: 'true'
TARGETS: ./tests/npm
TEAM: network-performance-monitoring Changes Summary
ℹ️ Diff available in the job log. |
Co-authored-by: Nicolas Schweitzer <nicolas.schweitzer@datadoghq.com>
Co-authored-by: Nicolas Schweitzer <nicolas.schweitzer@datadoghq.com>
Co-authored-by: Nicolas Schweitzer <nicolas.schweitzer@datadoghq.com>
Co-authored-by: Nicolas Schweitzer <nicolas.schweitzer@datadoghq.com>
Co-authored-by: Nicolas Schweitzer <nicolas.schweitzer@datadoghq.com>
…atadog-agent into kfairise/test-early-create-eks
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM, thanks for the updates
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
lgtm
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
👍 for container-platform files
…/test-early-create-eks
…atadog-agent into kfairise/test-early-create-eks
What does this PR do?
Add options and configure CI to run EKS test with a previous job that early start the creation of EKS cluster.
It makes things way more complex but it allows us to reduce by 30 minutes the duration of the
containers
EKS test which shorten by 30 minutes the duration of the whole CI.Per se, it does not reduce the overall duration of the EKS jobs, but it allows to start the setup earlier in the CI which should reduce the overall duration of the CI by 30 minutes since EKS jobs are in the critical path
Needs this change on test-infra to work because containers tests does not use the Provider defined on datadog-agent repository
The overall idea is to have two jobs that share the same Pulumi stack, the first job will apply the stack and stop when we reach a point defined in the run function. The second call that is made when all the dependencies are ready will run the complete run fonction and execute the tests. All the resources created in the first call are reused.
With that we're still able to rerun the container test manually, recreating the stack from scratch
Motivation
Speed up the CI by creating earlier the resources that can be created
Additional Notes
In the future we plan to remove the cleanup part from the test itself and instead mark the stack for deletion. And let an external workflow to handle the deletion. Which would allow us to remove the cleanup job as well
Possible Drawbacks / Trade-offs
The setup is quite manual and would require us to create the two jobs everytime we want to speed EKS tests. However we do not expect a lot of EKS tests, as we rather recommend to use Kind. So it should be fine
Describe how to test/QA your changes