Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

E2E: cover kamu-cli commands #882

Open
wants to merge 13 commits into
base: master
Choose a base branch
from
Open

E2E: cover kamu-cli commands #882

wants to merge 13 commits into from

Conversation

s373r
Copy link
Member

@s373r s373r commented Oct 8, 2024

Description

Closes: #877 , #898

PRs

Added

  • Added (or expanded) E2E tests for:
    • kamu config command
    • kamu init command
    • kamu add command
    • kamu rename command
    • kamu ingest command
    • kamu inspect command
    • kamu log command
    • kamu new command
    • kamu reset command
    • kamu search command
    • kamu sql command
    • kamu system gc command
    • kamu system info command
    • kamu system diagnose command
    • kamu tail command
    • kamu login command
    • kamu logout command
    • kamu push command
    • kamu pull command
  • E2E: HTTP middleware is implemented, which improves stability of E2E tests

Fixed

  • kamu add: fixed behavior when using --stdin and --name arguments

Checklist before requesting a review

s373r and others added 13 commits October 16, 2024 00:29
* KamuCliApiServerHarnessOptions::with_kamu_config(): introduce

* E2E: cover "kamu config" command

* E2E, test_init_in_an_existing_workspace(): add

* E2E, test_config_set_value(): respect potential CI agent podman configuration

* Updated yanked crate: futures-channel

* E2E, test_config_get_with_default(): respect potential CI agent podman configuration

* kamu-cli-e2e-common: extract "player-scores" & "leaderboard" datasets into statics

* assert_cmd: add patch

* E2E, test_add_dataset_from_stdin(): add

* E2E, test_add_dataset_from_stdin(): add

* AddCommand::validate_args(): fix "name" arg processing

* E2E, test_add_dataset_with_name(): mix add commands

* E2E, test_add_dataset_with_replace(): add

* Interact: fix if is_tty

* CHANGELOG.md: update

* E2E, test_delete_dataset(): add

* E2E, test_delete_dataset_recursive(): add

* E2E, test_delete_dataset_all(): add

* E2E, test_rename_dataset(): add

* assert_cmd: use patch (now from the kamu-data org)

* kamu-cli-puppet: remove unused dep

* Extracted to the separate PR
…etc command tests (#883)

* E2E, ingest_data_to_player_scores_from_stdio(): add

* clippy fixes

* E2E, test_ingest_from_stdin(): also check data via the tail command

* E2E, ingest_data_to_player_scores_from_stdio(): check data for all ingest steps

* E2E, test_ingest_recursive(): add... a part of the test

* E2E, test_ingest_with_source_name(): add

* E2E, test_inspect_lineage(): add

* IngestCommand: add a todo

* DATASET_ROOT_PLAYER_SCORES_INGEST_DATA_NDJSON_CHUNK_{1,2,3}: stabilize tests

* Rename: LineageCommand -> InspectLineageCommand

* E2E: stabilize ingest command tests

* KamuCliPuppetExt::list_blocks(): add

* E2E, test_inspect_query(): add

* E2E, test_inspect_schema(): add

* kamu-cli, Login: fix a doc string typo

* E2E, test_log(): add

* E2E, test_new_{root,derivative}(): add

* E2E, test_reset(): add

* cli-reference.md: update

* test_repo_alias_command -> test_repo_command

* E2E, KamuApiServerClientExt::ingest_data(): add

* E2E, test_search_multi_user(): add

* E2E, test_search_by_name(): add

* E2E, test_search_by_repo(): add

* E2E, test_sql_command(): add

* E2E, test_gc(): add

* E2E, test_system_info(): add

* E2E, test_system_diagnose(): add

* CI: test fixes

* assert_ingest_data_to_player_scores_from_stdio(): stabilize rows order

* assert_ingest_data_to_player_scores_from_stdio(): stabilize rows order [2]

* test_sql_command(): use the default name

* assert_ingest_data_to_player_scores_from_stdio(): stabilize rows order [3]

* CHANGELOG.md: update

* CI: windows build fixes

* E2E, test_tail(): add

* assert_ingest_data_to_player_scores_from_stdio(): stabilize rows order [4]

* assert_ingest_data_to_player_scores_from_stdio(): stabilize rows order [5]

* pretty_assertions::assert_eq!(): expected first, actual after

* Remove extra files

* E2E: cover "kamu login" / "kamu logout" commands
* Add query commitments example

* Release (patch) `0.204.5`: fix `--yes / -y` flag: fixed when working from a TTY (#881)

* Interact: fix if is_tty

* CHANGELOG.md: update

* Release (patch): 0.204.5

* Updated yanked crate (futures-util to 0.3.31)

* CI: Fixes `kamu-base-with-data-mt` image builds (#885)

* Images, kamu-base-with-data-mt: add "kamu" to predefined users

* Images, kamu-base-with-data-mt: init-workspace.py use .kamuconfig

* CHANGELOG.md: update

* Add push pull tests

* Add s3 tests

* Remove unused code

* Test transform engine error

* Revert check

* Add container extra groups

* Add verify and compact commands

* Fix grammar

* Fix review commets

* Revert "Merge branch 'master' into chore/push-and-pul-command-e2e-tests"

This reverts commit a4ec39c, reversing
changes made to 1130fa5.

* Clean code

* Fix review comments. Iter 2

* Revert ToDo comment

---------

Co-authored-by: Sergii Mikhtoniuk <mikhtoniuk@gmail.com>
Co-authored-by: Dima Pristupa <dima@pristupa.dev>
* E2E, e2e_middleware_fn(): implement

* kamu-adapter-http: hide E2E things behind the "e2e" feature gate

* query_handler_post_v2(): fix a typo

* kamu-cli: integrate the E2E middleware

* CHANGELOG.md: update
…on (#903)

* E2E: test_smart_pull_derivative(): use the tail command

* E2E: run postgres tests one by one

* E2E: test_smart_pull_derivative(): stabilize order

* E2E: wait_for_flows_to_finish(): increase takes count

* E2E, test_ingest_command: stabilize order without the "match_time" changing

* E2E, test_log_command: stabilize order without the "match_time" changing

* E2E, test_sql_command: stabilize order without the "match_time" changing

* E2E, test_tail_command: stabilize order without the "match_time" changing

* E2E, test_dataset_trigger_flow(): stabilize order without the "match_time" changing

* E2E, test_smart_transfer_protocol: stabilize order without the "match_time" changing
* E2E, test_rest_api_request_dataset_tail(): use a const instead of a hardcoded value

* E2E, test_verify_command: use a consts instead of hardcoded values

* E2E: KamuApiServerClient::get_node_url(): absorb

* CI: mark postgres as flaky but run with 8 threads

* Tests: add flaky "risingwave" group

* E2E, test_dataset_trigger_flow(): add "risingwave" test-group

* CI: go not specify threads for postgres

* E2E, KamuCliPuppetExt::assert_player_scores_dataset_data(): extract

* E2E, KamuCliPuppetExt::assert_success_command_execution(): absorb

* E2E, KamuCliPuppetExt::assert_failure_command_execution(): absorb

* E2E, assert_ingest_data_to_player_scores_from_stdio(): use assert_success_command_execution_with_input()

* KamuCliPuppetExt::assert_player_scores_dataset_data(): use assert_success_command_execution()

* E2E, test_config_command: use new methods

* E2E, test_inspect_command: use new methods

* E2E, test_search_command: use new methods

* E2E, test_sql_command: use new methods

* E2E, test_tail_command: use new methods

* E2E, test_system_api_server_gql_query: use new methods

* E2E, test_add_command: use new methods

* KamuCliPuppetExt: use iter for maybe_expected_stderr

* E2E, test_compact_command: use new methods

* E2E, test_delete_command: use new methods

* E2E, test_ingest_command: use new methods

* E2E, test_init_command: use new methods

* E2E, test_login_command: use new methods

* E2E, test_new_command: use new methods

* E2E, test_rename_command: use new methods

* E2E, test_reset_command: use new methods

* E2E, test_smart_transfer_protocol: use new methods

* E2E, test_system_gc_command: use new methods

* E2E, test_verify_command: use new methods

* E2E, test_workspace_svc: use new methods

* KamuCliPuppetExt::assert_failure_command_execution_with_input(): implement
@s373r s373r marked this pull request as ready for review October 15, 2024 21:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

E2E: cover kamu-cli commands
2 participants