Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add LLM model server metrics #1103

Merged
merged 15 commits into from
Jun 27, 2024
Merged

Add LLM model server metrics #1103

merged 15 commits into from
Jun 27, 2024

Commits on Jun 20, 2024

  1. Add LLM model server metrics 

    This change adds common model server metrics that we want to standardize
on. It starts of with two common latency metrics - time per output token
and time to first token.
    @achandrasekar
    achandrasekar committed Jun 20, 2024
    Configuration menu
    Copy the full SHA
    c277263 View commit details
    Browse the repository at this point in the history

  2. Add changelog

    @achandrasekar
    achandrasekar committed Jun 20, 2024
    Configuration menu
    Copy the full SHA
    3b81d8f View commit details
    Browse the repository at this point in the history

  3. Address typos and clarify description

    @achandrasekar
    achandrasekar committed Jun 20, 2024
    Configuration menu
    Copy the full SHA
    57f9e89 View commit details
    Browse the repository at this point in the history

  4. Add request_duration metric and fix description of other ones

    @achandrasekar
    achandrasekar committed Jun 20, 2024
    Configuration menu
    Copy the full SHA
    479d2dc View commit details
    Browse the repository at this point in the history

  5. Add markdown toc

    @achandrasekar
    achandrasekar committed Jun 20, 2024
    Configuration menu
    Copy the full SHA
    1d2341f View commit details
    Browse the repository at this point in the history

  6. Update .chloggen/1102.yaml to fix typo 

    Co-authored-by: Liudmila Molkova <limolkova@microsoft.com>
    @achandrasekar @lmolkova
    achandrasekar and lmolkova committed Jun 20, 2024
    Configuration menu
    Copy the full SHA
    b3117f4 View commit details
    Browse the repository at this point in the history

  7. Update docs/gen-ai/gen-ai-metrics.md metric description 

    Co-authored-by: Liudmila Molkova <limolkova@microsoft.com>
    @achandrasekar @lmolkova
    achandrasekar and lmolkova committed Jun 20, 2024
    Configuration menu
    Copy the full SHA
    8e10b93 View commit details
    Browse the repository at this point in the history

  8. Drop latency from the metric names

    @achandrasekar
    achandrasekar committed Jun 20, 2024
    Configuration menu
    Copy the full SHA
    c250635 View commit details
    Browse the repository at this point in the history

  9. Render attribute table for the metrics

    @achandrasekar
    achandrasekar committed Jun 20, 2024
    Configuration menu
    Copy the full SHA
    993411a View commit details
    Browse the repository at this point in the history

  10. Addressed error type, buckets and descriptions

    @achandrasekar
    achandrasekar committed Jun 20, 2024
    Configuration menu
    Copy the full SHA
    28548d8 View commit details
    Browse the repository at this point in the history

  11. Fix formatting after addressing conflicts

    @achandrasekar
    achandrasekar committed Jun 20, 2024
    Configuration menu
    Copy the full SHA
    2873d9f View commit details
    Browse the repository at this point in the history

Commits on Jun 21, 2024

  1. Apply suggestions from code review 

    Co-authored-by: Drew Robbins <drew@drewby.com>
Co-authored-by: Liudmila Molkova <limolkova@microsoft.com>
    @achandrasekar @drewby @lmolkova
    3 people authored Jun 21, 2024
    Configuration menu
    Copy the full SHA
    2c8c835 View commit details
    Browse the repository at this point in the history

  2. Fix formatting and naming

    @achandrasekar
    achandrasekar committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    9cd3b0d View commit details
    Browse the repository at this point in the history

Commits on Jun 26, 2024

  1. Merge branch 'main' into model-server-metrics

    @joaopgrassi
    joaopgrassi authored Jun 26, 2024
    Configuration menu
    Copy the full SHA
    a955cde View commit details
    Browse the repository at this point in the history

Commits on Jun 27, 2024

  1. Merge branch 'main' into model-server-metrics

    @joaopgrassi
    joaopgrassi authored Jun 27, 2024
    Configuration menu
    Copy the full SHA
    0cd2bdd View commit details
    Browse the repository at this point in the history