Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Add GRPC error codes to GRPC streaming if enabled by user. #7499

Merged
merged 35 commits into from
Aug 16, 2024
Merged
Changes from 1 commit
Commits
Show all changes
35 commits
Select commit Hold shift + click to select a range
33c1e93
Park
indrajit96 Jul 25, 2024
3affd3a
Park
indrajit96 Jul 30, 2024
7f86c6a
Working Set
indrajit96 Jul 31, 2024
b65fd74
Working Set
indrajit96 Aug 1, 2024
36e461f
Working Set
indrajit96 Aug 1, 2024
bd549b1
Working Set
indrajit96 Aug 1, 2024
85ccd72
Tests Added
indrajit96 Aug 2, 2024
faa742c
Merge branch 'main' into ibhosale_grpc_streaming
indrajit96 Aug 2, 2024
37b15e8
Clean up
indrajit96 Aug 2, 2024
a65f8c3
Tests updated
indrajit96 Aug 5, 2024
080985c
Zombie request fixed
indrajit96 Aug 6, 2024
cc34d41
Pre Commit fixed
indrajit96 Aug 6, 2024
40344d5
Review Comments cleaned up, crash fixed in multi threading
indrajit96 Aug 7, 2024
f40f695
Review Comments fixed
indrajit96 Aug 7, 2024
0792bc1
Pre-Commit fixed
indrajit96 Aug 7, 2024
cdd60bf
Park
indrajit96 Aug 8, 2024
6661f21
Simpler design piggyback on NotifywhenDone()
indrajit96 Aug 9, 2024
6342524
Merge branch 'main' into ibhosale_grpc_streaming
indrajit96 Aug 9, 2024
22e5359
Cleanup unwanted states from old design
indrajit96 Aug 9, 2024
a31ba09
Improved tests around triton_grpc_error mode
indrajit96 Aug 12, 2024
af451a2
Comments resolved
indrajit96 Aug 14, 2024
0cb7db0
Comments resolved
indrajit96 Aug 14, 2024
8ea2647
New class gRPCErrorTracker created
indrajit96 Aug 14, 2024
e8c3242
Docs Updated
indrajit96 Aug 14, 2024
72097e3
Pipeline test
indrajit96 Aug 15, 2024
350af25
Resolve Unused local variable warning
indrajit96 Aug 15, 2024
e473f29
GRPC Cleanup tests updated for triton grpc error
indrajit96 Aug 16, 2024
370c449
Revert "GRPC Cleanup tests updated for triton grpc error"
indrajit96 Aug 16, 2024
b87c3fc
GRPC Cleanup tests updated for triton grpc error
indrajit96 Aug 16, 2024
cb548ff
Pre-Commit format
indrajit96 Aug 16, 2024
1b6b3a7
Devel build fix
indrajit96 Aug 16, 2024
70ce279
Streamline new tests
indrajit96 Aug 16, 2024
631b352
Merge branch 'main' into ibhosale_grpc_streaming
indrajit96 Aug 16, 2024
887aaa2
PR comments fixed and main merged
indrajit96 Aug 16, 2024
0e7670c
DockerFile fixed
indrajit96 Aug 16, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Docs Updated
  • Loading branch information
indrajit96 committed Aug 14, 2024
commit e8c3242b5e902ab983bbac8d9fc2fef4f2fa16a0
10 changes: 10 additions & 0 deletions docs/customization_guide/inference_protocols.md
Original file line number Diff line number Diff line change
Expand Up @@ -115,6 +115,16 @@ These options can be used to configure the KeepAlive settings:

For client-side documentation, see [Client-Side GRPC KeepAlive](https://github.com/triton-inference-server/client/blob/main/README.md#grpc-keepalive).

#### GRPC Status Codes

Triton implements GRPC error handling for streaming requests when a specific flag is enabled through headers. Upon encountering an error, Triton returns the appropriate GRPC error code and subsequently closes the stream.

* `triton_grpc_error` : The header value needs to be set to true while starting the stream.

GRPC status codes can be used for better visibility and monitoring. For more details, see [gRPC Status Codes](https://grpc.io/docs/guides/status-codes/)

For client-side documentation, see [Client-Side GRPC Status Codes](https://github.com/triton-inference-server/client/tree/main#GRPC-Status-Codes)

### Limit Endpoint Access (BETA)

Triton users may want to restrict access to protocols or APIs that are
Expand Down
Loading