pip installable comfyui, single file .exe, multi-backend / distributed comfyui, pip installable plugins, rich API, OpenAPI specs

[doctorpangloss]

A fully featured update to ComfyUI

This fork allows you to

• install ComfyUI using pip install git+https://github.com/hiddenswitch/ComfyUI.git. You never need to think about installing torch again.
• build a ComfyUI .exe using python -m PyInstaller --onefile --noupx -n ComfyUI --add-data="comfy/;comfy/" --paths . --paths comfy/cmd main.py
• start comfyui using comfyui with lots of useful options you can find in --help
• run a distributed comfyui, separating the frontend from the backend, with support for unlimited backends. Start RabbitMQ, then comfyui-worker --distributed-queue-connection-uri=amqp://guest:guest@rabbitmq-server --cwd //shared/network/path on as many machines as you'd like; comfyui --distributed-queue-roles=frontend --distributed-queue-connection-uri=amqp://guest:guest@rabbitmq-server --cwd //shared/network/path on others.
• author plugins with a setup.py, and install new nodes or web extensions with pip install git+https://github.com/some/repository.git. No more messing around with requirements.txt or manually resolving dependency conflicts.
• build your own applications using a rich set of approaches, such as EmbeddedComfyClient to run comfyui prompts directly in your python application without a backend; AsyncRemoteComfyClient to connect to a remote server, or a fully-featured OpenAPI 3.1 spec to make a rich UI.

Previous Discussion

This includes the setup.py changes: (resolves #1613 #1342 #188)

• setup.py now works
• always installs the correct gpu accelerated pytorch for your platform, resolving issues like KSampler failed again #1759 RuntimeError in import_utils.py #1753 Broken : × python setup.py egg_info did not run successfully. #1737 OSError: [WinError 193] #1694 Incompatible architecture on M1 #1164 After Ventura 13.5 update on MacBook I am getting error #1066 Can't make package for torchvision rocm steam deck distrobox #1130 Dependency installation fails: No matching distribution found for torchdiffeq #813
• provides a way for other packages to export custom nodes. they should have entry points of the form

setup.py

setup(...,
entry_points={
        # key is required
        'comfyui.custom_nodes': [
            'mypackagename = module.root.to.recursively.scan.for.nodes',
        ],
    })

all .py files with

NODE_CLASS_MAPPINGS = {
    "BinaryPreprocessor": Binary_Preprocessor
}
NODE_DISPLAY_NAME_MAPPINGS = {
    "BinaryPreprocessor": "Binary Lines"
}

that are in a package (i.e. a directory with an __init__.py) in module.root.to.recursively.scan.for.nodes or deeper will be recognized by comfyui as providing nodes.

since you will be using pip to install custom nodes this way, you will get dependency resolution automatically, so you never have to pip install -r requirements.txt or similar garbage. this will deprecate / obsolete ComfyUI-Manager's functionality, you can use any pip package manager UI (mostly resolves #1773). this also fixes issues like #1678 #1665 #1596 #1385 #1373 if adopted

you would also be able to add the comfyui git hash and custom nodes packages by git+commit or name in the metadata for maximum reproducibility.

This PR also creates two new api endpoints documented in the openapi.yaml. These are ergonomic for common usage like batches or using from other applications.

test using the instructions from: https://github.com/hiddenswitch/ComfyUI/tree/master#manual-install-windows-linux-macos

also copied here to improve comprehension of the purpose of this PR

Manual Install (Windows, Linux, macOS)

You must have Python 3.11 or 3.10 installed. 3.12 is not yet supported. On Windows, download the latest Python from their website. You can also directly download 3.11.4 here..

On macOS, install exactly Python 3.11 using brew, which you can download from https://brew.sh, using this command: brew install python@3.11. Do not use 3.9 or older, and do not use 3.12 or newer. Its compatibility with Stable Diffusion in both directions is broken.

1. Create a virtual environment:

   python -m virtualenv venv

2. Activate it on
   Windows (PowerShell):

   Set-ExecutionPolicy Unrestricted -Scope Process
   & .\venv\Scripts\activate.ps1

   Linux and macOS

   source ./venv/bin/activate

3. Then, run the following command to install comfyui into your current environment. This will correctly select the version of pytorch that matches the GPU on your machine (NVIDIA or CPU on Windows, NVIDIA AMD or CPU on Linux):

   pip install git+https://github.com/hiddenswitch/ComfyUI.git

4. To run the web server:

   comfyui

   To generate python OpenAPI models:

   comfyui-openapi-gen

Manual Install (Windows, Linux, macOS) For Development

1. Clone this repo:

   git clone https://github.com/comfyanonymous/ComfyUI.git
   cd ComfyUI

2. Put your Stable Diffusion checkpoints (the huge ckpt/safetensors files) into the models/checkpoints folder. You can download SD v1.5 using the following command:

   curl -L https://huggingface.co/runwayml/stable-diffusion-v1-5/resolve/main/v1-5-pruned-emaonly.ckpt -o ./models/checkpoints/v1-5-pruned-emaonly.ckpt

3. Put your VAE into the models/vae folder.

4. (Optional) Create a virtual environment:

   1. Create an environment:

      python -m virtualenv venv

   2. Activate it:

      Windows (PowerShell):

      Set-ExecutionPolicy Unrestricted -Scope Process
      & .\venv\Scripts\activate.ps1

      Linux and macOS

      source ./venv/bin/activate

5. Then, run the following command to install comfyui into your current environment. This will correctly select the version of pytorch that matches the GPU on your machine (NVIDIA or CPU on Windows, NVIDIA AMD or CPU on Linux):

   pip install -e .

6. To run the web server:

   comfyui

   To generate python OpenAPI models:

   comfyui-openapi-gen

   You can use comfyui as an API. Visit the OpenAPI specification. This file can be used to generate typed clients for your preferred language.

[tildebyte]

This looks really good (worked well for me).

[doctorpangloss]

This is now tested on fairly complex setups except AMD DirectML on Windows. I will try that on my AMD Windows station later. Additionally, PyTorch 2.1+ nightlies are now the default for all accelerators I tested (Windows NVIDIA, Windows CPU, Linux NVIDIA, Linux AMD, macOS Apple Silicon). This leaves Windows AMD and macOS AMD to test.

[tildebyte]

PyTorch 2.1+ nightlies

Looks like for NVidia, I currently get torch@https://download.pytorch.org/whl/cu118/torch-2.0.1%2Bcu118-cp311-cp311-win_amd64.whl (the url in setup.py is for torch 2.0.1 and CUDA 11.8) when I run pip install -e ., not 2.1+nightly (unless maybe there's a dependency which I'm not seeing which causes torch==2.1)

[doctorpangloss]

That's the error! I accidentally left out that commit, I'll tie up my other work shortly and tidy this up.