Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Failing ES Promotion: FTR Configs #32 / Fleet Endpoints fleet policy secrets Should correctly create the policy with secrets #162732

Closed
mistic opened this issue Jul 28, 2023 · 4 comments · Fixed by #163428
Closed

Failing ES Promotion: FTR Configs #32 / Fleet Endpoints fleet policy secrets Should correctly create the policy with secrets #162732

mistic opened this issue Jul 28, 2023 · 4 comments · Fixed by #163428
Assignees
@hop-dev
Labels
skipped-test Team:Fleet Team label for Observability Data Collection Fleet team v8.10.0

Comments

@mistic
Copy link
Member

mistic commented Jul 28, 2023

X-Pack Fleet API Integration Tests
x-pack/test/fleet_api_integration/apis/policy_secrets.ts

Fleet Endpoints fleet policy secrets Should correctly create the policy with secrets

This failure is preventing the promotion of the current Elasticsearch snapshot.

For more information on the Elasticsearch snapshot process including how to reproduce using the unverified ES build please read the failed promotion annotation. Other important information can be found at:

Error: expected 200 "OK", got 400 "Bad Request"
    at Test._assertStatus (node_modules/supertest/lib/test.js:268:12)
    at Test._assertFunction (node_modules/supertest/lib/test.js:283:11)
    at Test.assert (node_modules/supertest/lib/test.js:173:18)
    at localAssert (node_modules/supertest/lib/test.js:131:12)
    at /var/lib/buildkite-agent/builds/kb-n2-4-spot-54872ae440cf6bd7/elastic/kibana-elasticsearch-snapshot-verify/kibana/node_modules/supertest/lib/test.js:128:5
    at Test.Request.callback (node_modules/superagent/lib/node/index.js:728:3)
    at /var/lib/buildkite-agent/builds/kb-n2-4-spot-54872ae440cf6bd7/elastic/kibana-elasticsearch-snapshot-verify/kibana/node_modules/superagent/lib/node/index.js:916:18
    at Stream.<anonymous> (node_modules/superagent/lib/node/parsers/json.js:19:7)
    at Stream.emit (node:events:514:28)
    at Unzip.<anonymous> (node_modules/superagent/lib/node/unzip.js:55:12)
    at Unzip.emit (node:events:514:28)
    at endReadableNT (node:internal/streams/readable:1359:12)
    at processTicksAndRejections (node:internal/process/task_queues:82:21)
@mistic mistic added blocker skipped-test failed-es-promotion Team:Fleet Team label for Observability Data Collection Fleet team v8.10.0 labels Jul 28, 2023
@elasticmachine
Copy link
Contributor

Pinging @elastic/fleet (Team:Fleet)

mistic added a commit that referenced this issue Jul 28, 2023
@mistic
skip failing es promotion (#162732)
b183053
@mistic
Copy link
Member Author

mistic commented Jul 28, 2023

Skipped.

main: b183053

@sophiec20
Copy link
Contributor

This test failure appears to be directly related to a newer version of Elasticsearch. This test was skipped in order to unblock the Elasticsearch snapshot promotion.

So that we can be ready to release from main, we ask for failed-es-promotion issues to be treated with greater urgency. Please acknowledge and triage this issue.

ThomThomson pushed a commit to ThomThomson/kibana that referenced this issue Aug 1, 2023
@mistic
skip failing es promotion (elastic#162732)
d0fe075
@joshdover
Copy link
Contributor

This affects a feature that is behind a feature flag and is still in development. We are going to remove the blocker labels from this issue and will fix and unskip the test as part of the ongoing development.

@joshdover joshdover mentioned this issue Aug 2, 2023
Closed
@hop-dev hop-dev self-assigned this Aug 3, 2023
@criamico criamico mentioned this issue Aug 8, 2023
Merged
criamico added a commit that referenced this issue Aug 8, 2023
@criamico @kibanamachine
[Fleet] Use new Fleet Secrets ES APIs instead of reading/writing secr…
c249c30 
…ets index (#163075)

Closes #162915

## Summary
Replace direct calls to Fleet Secrets index with new API calls
introduced with elastic/elasticsearch#97728

### New ES secrets APIs:
```
POST /_fleet/secret/
{
  "value": "<secret value>"
}

// Returns the id of the created secret
{
  "id": "<secret_id>"
}

DELETE /_fleet/secret/<secret_id>

// returns 
{
  "deleted": true
}
```

NOTE: I tried running the secrets integration tests in
#162732 but there is some ES
error that I'm not sure how to address. I think that the test can be
worked on separately

### Testing

Testing steps are the exact same as
#157176:
- Start EPR locally loading the `Secrets` test package from Kibana:

```
docker run -p 8080:8080 -v /Users/<YOUR_PATH>/kibana/x-pack/test/fleet_api_integration/apis/fixtures/test_packages:/packages/test-packages -v /Users/<YOUR_PATH>/kibana/x-pack/test/fleet_api_integration/apis/fixtures/package_registry_config.yml:/package-registry/config.yml docker.elastic.co/package-registry/package-registry:main
```
- Point `kibana.dev.yml` to local EPR:
```
  xpack.fleet.registryUrl: http://localhost:8080
```
- Enable the secrets feature flag `secretsStorage`
- Start kibana and navigate to `integrations`, install `Secrets`
package.
- It should create and edit the package policy successfully

<img width="1800" alt="Screenshot 2023-08-08 at 16 26 52"
src="https://github.com/elastic/kibana/assets/16084106/5e2b77d9-71a9-4c5f-8b3b-5fc6546d562f">

- The yml policy should have the redacted secrets and secrets ids:

<img width="771" alt="Screenshot 2023-08-08 at 15 43 22"
src="https://github.com/elastic/kibana/assets/16084106/7db22c6b-b0db-4eb6-bc68-7174374c9c74">

---------

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
@hop-dev hop-dev mentioned this issue Aug 8, 2023
Merged
bryce-b pushed a commit to bryce-b/kibana that referenced this issue Aug 9, 2023
@criamico @kibanamachine @bryce-b
[Fleet] Use new Fleet Secrets ES APIs instead of reading/writing secr…
00c6208 
…ets index (elastic#163075)

Closes elastic#162915

## Summary
Replace direct calls to Fleet Secrets index with new API calls
introduced with elastic/elasticsearch#97728

### New ES secrets APIs:
```
POST /_fleet/secret/
{
  "value": "<secret value>"
}

// Returns the id of the created secret
{
  "id": "<secret_id>"
}

DELETE /_fleet/secret/<secret_id>

// returns 
{
  "deleted": true
}
```

NOTE: I tried running the secrets integration tests in
elastic#162732 but there is some ES
error that I'm not sure how to address. I think that the test can be
worked on separately

### Testing

Testing steps are the exact same as
elastic#157176:
- Start EPR locally loading the `Secrets` test package from Kibana:

```
docker run -p 8080:8080 -v /Users/<YOUR_PATH>/kibana/x-pack/test/fleet_api_integration/apis/fixtures/test_packages:/packages/test-packages -v /Users/<YOUR_PATH>/kibana/x-pack/test/fleet_api_integration/apis/fixtures/package_registry_config.yml:/package-registry/config.yml docker.elastic.co/package-registry/package-registry:main
```
- Point `kibana.dev.yml` to local EPR:
```
  xpack.fleet.registryUrl: http://localhost:8080
```
- Enable the secrets feature flag `secretsStorage`
- Start kibana and navigate to `integrations`, install `Secrets`
package.
- It should create and edit the package policy successfully

<img width="1800" alt="Screenshot 2023-08-08 at 16 26 52"
src="https://github.com/elastic/kibana/assets/16084106/5e2b77d9-71a9-4c5f-8b3b-5fc6546d562f">

- The yml policy should have the redacted secrets and secrets ids:

<img width="771" alt="Screenshot 2023-08-08 at 15 43 22"
src="https://github.com/elastic/kibana/assets/16084106/7db22c6b-b0db-4eb6-bc68-7174374c9c74">

---------

Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
hop-dev added a commit that referenced this issue Aug 10, 2023
@hop-dev @kibanamachine
[Fleet] Re-enable and fix Fleet policy secret integration tests (#163428
94432d8 
)

## Summary

Closes #162732 
Closes #157503

Wanted to sneak this in before we move over to the internal index, I
have tidied the tests a bit to make that transition easier.

Since we restricted the fleet service account permissions, we can no
longer use a test index for the secret tests. The test index was added
while .fleet-secrets didn't exist so I have switched to using the real
index.

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@hop-dev hop-dev

Labels
skipped-test Team:Fleet Team label for Observability Data Collection Fleet team v8.10.0
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

[Fleet] Re-enable and fix Fleet policy secret integration tests hop-dev/kibana
5 participants
@joshdover @hop-dev @sophiec20 @mistic @elasticmachine