Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

ggml : add RPC backend #6829

Merged
merged 19 commits into from
May 14, 2024
Merged

ggml : add RPC backend #6829

merged 19 commits into from
May 14, 2024
+1,395 −98

Commits on May 14, 2024

  1. ggml : add RPC backend 

    The RPC backend proxies all operations to a remote server which runs a
regular backend (CPU, CUDA, Metal, etc).
    @rgerganov
    rgerganov committed May 14, 2024
    Configuration menu
    Copy the full SHA
    bfdd4f4 View commit details
    Browse the repository at this point in the history

  2. set TCP_NODELAY

    @rgerganov
    rgerganov committed May 14, 2024
    Configuration menu
    Copy the full SHA
    e539b45 View commit details
    Browse the repository at this point in the history

  3. add CI workflows

    @rgerganov
    rgerganov committed May 14, 2024
    Configuration menu
    Copy the full SHA
    3562c33 View commit details
    Browse the repository at this point in the history

  4. Address review comments

    @rgerganov
    rgerganov committed May 14, 2024
    Configuration menu
    Copy the full SHA
    c854687 View commit details
    Browse the repository at this point in the history

  5. fix warning

    @rgerganov
    rgerganov committed May 14, 2024
    Configuration menu
    Copy the full SHA
    95c16c2 View commit details
    Browse the repository at this point in the history

  6. implement llama_max_devices() for RPC

    @rgerganov
    rgerganov committed May 14, 2024
    Configuration menu
    Copy the full SHA
    654c1cc View commit details
    Browse the repository at this point in the history

  7. Address review comments

    @rgerganov
    rgerganov committed May 14, 2024
    Configuration menu
    Copy the full SHA
    cddbf97 View commit details
    Browse the repository at this point in the history

  8. Address review comments

    @rgerganov
    rgerganov committed May 14, 2024
    Configuration menu
    Copy the full SHA
    dfadd1a View commit details
    Browse the repository at this point in the history

  9. wrap sockfd into a struct

    @rgerganov
    rgerganov committed May 14, 2024
    Configuration menu
    Copy the full SHA
    ef9be32 View commit details
    Browse the repository at this point in the history

  10. implement get_alignment and get_max_size

    @rgerganov
    rgerganov committed May 14, 2024
    Configuration menu
    Copy the full SHA
    7a963c3 View commit details
    Browse the repository at this point in the history

  11. add get_device_memory

    @rgerganov
    rgerganov committed May 14, 2024
    Configuration menu
    Copy the full SHA
    0b5e8a7 View commit details
    Browse the repository at this point in the history

  12. fix warning

    @rgerganov
    rgerganov committed May 14, 2024
    Configuration menu
    Copy the full SHA
    3d55181 View commit details
    Browse the repository at this point in the history

  13. win32 support

    @rgerganov
    rgerganov committed May 14, 2024
    Configuration menu
    Copy the full SHA
    7c00fd5 View commit details
    Browse the repository at this point in the history

  14. add README

    @rgerganov
    rgerganov committed May 14, 2024
    Configuration menu
    Copy the full SHA
    7975f43 View commit details
    Browse the repository at this point in the history

  15. readme : trim trailing whitespace

    @rgerganov
    rgerganov committed May 14, 2024
    Configuration menu
    Copy the full SHA
    df54ada View commit details
    Browse the repository at this point in the history

  16. Address review comments

    @rgerganov
    rgerganov committed May 14, 2024
    Configuration menu
    Copy the full SHA
    b4bf42a View commit details
    Browse the repository at this point in the history

  17. win32 fix

    @rgerganov
    rgerganov committed May 14, 2024
    Configuration menu
    Copy the full SHA
    cfd5d3e View commit details
    Browse the repository at this point in the history

  18. Address review comments

    @rgerganov
    rgerganov committed May 14, 2024
    Configuration menu
    Copy the full SHA
    caa37b2 View commit details
    Browse the repository at this point in the history

  19. fix compile warnings on macos

    @rgerganov
    rgerganov committed May 14, 2024
    Configuration menu
    Copy the full SHA
    1519cb4 View commit details
    Browse the repository at this point in the history