Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Kubernetes tracing and metrics improvements #3298

Merged
merged 4 commits into from
Feb 15, 2022
Merged

Conversation

steven-sheehy
Copy link
Member

@steven-sheehy steven-sheehy commented Feb 14, 2022

Description:

  • Add missing REST configuration to docs
  • Change REST API to log at info level by default to reduce log volume
  • Change REST API to generate a consistent trace log for all responses
  • Change Traefik to only log non-2xx requests to reduce log volume
  • Fix client IP not showing up in any trace logs
  • Fix Lettuce histogram metrics enabled by default and generating ~10K series
  • Fix monitor not tracing requests in some scenarios and add test coverage
  • Improve GCP Pub/Sub performance by reusing printer across transactions
  • Remove some REST API metric buckets to reduce number of overall time series

Related issue(s):

Notes for reviewer:

REST API output will now appear consistently in the below format:

INFO 9e4e7ca3 ::1 GET /api/v1/transactions?limit=2 in 37 ms: 200

Checklist

  • Documented (Code comments, README, etc.)
  • Tested (unit, integration, etc.)

Signed-off-by: Steven Sheehy <steven.sheehy@hedera.com>
@steven-sheehy steven-sheehy added enhancement Type: New feature P2 ops Tasks relating to network operations labels Feb 14, 2022
@steven-sheehy steven-sheehy added this to the 0.51.0 milestone Feb 14, 2022
@steven-sheehy steven-sheehy requested a review from a team February 14, 2022 17:39
@steven-sheehy steven-sheehy self-assigned this Feb 14, 2022
@codecov
Copy link

codecov bot commented Feb 14, 2022

Codecov Report

Merging #3298 (f64a8db) into main (743bca0) will increase coverage by 0.08%.
The diff coverage is 96.29%.

Impacted file tree graph

@@             Coverage Diff              @@
##               main    #3298      +/-   ##
============================================
+ Coverage     88.16%   88.25%   +0.08%     
- Complexity     2507     2517      +10     
============================================
  Files           502      502              
  Lines         14143    14164      +21     
  Branches       1306     1310       +4     
============================================
+ Hits          12469    12500      +31     
+ Misses         1365     1358       -7     
+ Partials        309      306       -3     
Impacted Files Coverage Δ
...a/com/hedera/mirror/web3/config/LoggingFilter.java 90.90% <ø> (+6.06%) ⬆️
...om/hedera/mirror/monitor/config/LoggingFilter.java 91.17% <95.00%> (+31.17%) ⬆️
...a/mirror/importer/config/MetricsConfiguration.java 91.66% <100.00%> (+1.19%) ⬆️
...mporter/parser/serializer/ProtoJsonSerializer.java 100.00% <100.00%> (ø)
constants.js 100.00% <0.00%> (ø)
middleware/requestHandler.js 100.00% <0.00%> (ø)
middleware/responseHandler.js 100.00% <0.00%> (ø)
middleware/httpErrorHandler.js 100.00% <0.00%> (ø)
...a/mirror/grpc/listener/CompositeTopicListener.java 83.33% <0.00%> (+4.16%) ⬆️
... and 1 more

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 743bca0...f64a8db. Read the comment docs.

Signed-off-by: Steven Sheehy <steven.sheehy@hedera.com>
Signed-off-by: Steven Sheehy <steven.sheehy@hedera.com>
Signed-off-by: Steven Sheehy <steven.sheehy@hedera.com>
@sonarcloud
Copy link

sonarcloud bot commented Feb 14, 2022

SonarCloud Quality Gate failed.    Quality Gate failed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 0 Code Smells

No Coverage information No Coverage information
10.2% 10.2% Duplication

@steven-sheehy steven-sheehy merged commit e5a826c into main Feb 15, 2022
@steven-sheehy steven-sheehy deleted the fix-client-ip branch February 15, 2022 18:29
matheus-dallrosa pushed a commit that referenced this pull request Feb 18, 2022
* Add missing REST configuration to docs
* Change REST API to log at info level by default to reduce log volume
* Change REST API to generate a consistent trace log for all responses
* Change Traefik to only log non-2xx requests to reduce log volume
* Fix client IP not showing up in any trace logs
* Fix Lettuce histogram metrics enabled by default and generating ~10K series
* Fix monitor not tracing requests in some scenarios and add test coverage
* Improve GCP Pub/Sub performance by reusing printer across transactions
* Remove some REST API metric buckets to reduce number of overall time series

Signed-off-by: Steven Sheehy <steven.sheehy@hedera.com>
Signed-off-by: Matheus DallRosa <matheus.dallrosa@swirlds.com>
matheus-dallrosa pushed a commit that referenced this pull request Feb 21, 2022
* Add missing REST configuration to docs
* Change REST API to log at info level by default to reduce log volume
* Change REST API to generate a consistent trace log for all responses
* Change Traefik to only log non-2xx requests to reduce log volume
* Fix client IP not showing up in any trace logs
* Fix Lettuce histogram metrics enabled by default and generating ~10K series
* Fix monitor not tracing requests in some scenarios and add test coverage
* Improve GCP Pub/Sub performance by reusing printer across transactions
* Remove some REST API metric buckets to reduce number of overall time series

Signed-off-by: Steven Sheehy <steven.sheehy@hedera.com>
Signed-off-by: Matheus DallRosa <matheus.dallrosa@swirlds.com>
matheus-dallrosa pushed a commit that referenced this pull request Feb 21, 2022
* Add missing REST configuration to docs
* Change REST API to log at info level by default to reduce log volume
* Change REST API to generate a consistent trace log for all responses
* Change Traefik to only log non-2xx requests to reduce log volume
* Fix client IP not showing up in any trace logs
* Fix Lettuce histogram metrics enabled by default and generating ~10K series
* Fix monitor not tracing requests in some scenarios and add test coverage
* Improve GCP Pub/Sub performance by reusing printer across transactions
* Remove some REST API metric buckets to reduce number of overall time series

Signed-off-by: Steven Sheehy <steven.sheehy@hedera.com>
Signed-off-by: Matheus DallRosa <matheus.dallrosa@swirlds.com>
matheus-dallrosa pushed a commit that referenced this pull request Feb 22, 2022
* Add missing REST configuration to docs
* Change REST API to log at info level by default to reduce log volume
* Change REST API to generate a consistent trace log for all responses
* Change Traefik to only log non-2xx requests to reduce log volume
* Fix client IP not showing up in any trace logs
* Fix Lettuce histogram metrics enabled by default and generating ~10K series
* Fix monitor not tracing requests in some scenarios and add test coverage
* Improve GCP Pub/Sub performance by reusing printer across transactions
* Remove some REST API metric buckets to reduce number of overall time series

Signed-off-by: Steven Sheehy <steven.sheehy@hedera.com>
Signed-off-by: Matheus DallRosa <matheus.dallrosa@swirlds.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Type: New feature ops Tasks relating to network operations P2
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants