Failing dependencies in example endpoints/model for image generation

[PrimeTimeTran]

LocalAI version:
Not sure where to find. Please specify and I'll update

Environment, CPU architecture, OS, and Version:

$ system_profiler SPHardwareDataType SPSoftwareDataType SPNetworkDataType

Hardware:

    Hardware Overview:

      Model Name: MacBook Pro
      Model Identifier: Mac15,7
      Model Number: Z1AF0019MLL/A
      Chip: Apple M3 Pro
      Total Number of Cores: 12 (6 performance and 6 efficiency)
      Memory: 18 GB
      System Firmware Version: 10151.81.1
      OS Loader Version: 10151.81.1
      Serial Number (system): DGXL7Y6L4M
      Hardware UUID: 9ADCBC07-4367-5F24-8527-18221CD37297
      Provisioning UDID: 00006030-000468910E28001C
      Activation Lock Status: Enabled

Software:

    System Software Overview:

      System Version: macOS 14.3 (23D56)
      Kernel Version: Darwin 23.3.0
      Boot Volume: Macintosh HD
      Boot Mode: Normal
      Computer Name: Loi’s MacBook Pro
      User Name: Loi Tran (future)
      Secure Virtual Memory: Enabled
      System Integrity Protection: Disabled
      Time since boot: 3 days, 21 hours, 1 minute

Network:

    Ethernet Adapter (en4):

      Type: Ethernet
      Hardware: Ethernet
      BSD Device Name: en4
      IPv4:
          Configuration Method: DHCP
      IPv6:
          Configuration Method: Automatic
      Ethernet:
          MAC Address: 92:ef:00:62:eb:ad
          Media Options: 
          Media Subtype: none
      Proxies:
          Exceptions List: *.local, 169.254/16
          FTP Passive Mode: Yes
      Service Order: 0

    Ethernet Adapter (en5):

      Type: Ethernet
      Hardware: Ethernet
      BSD Device Name: en5
      IPv4:
          Configuration Method: DHCP
      IPv6:
          Configuration Method: Automatic
      Ethernet:
          MAC Address: 92:ef:00:62:eb:ae
          Media Options: 
          Media Subtype: none
      Proxies:
          Exceptions List: *.local, 169.254/16
          FTP Passive Mode: Yes
      Service Order: 1

    USB 10/100/1000 LAN:

      Type: Ethernet
      Hardware: Ethernet
      BSD Device Name: en6
      IPv4:
          Configuration Method: DHCP
      IPv6:
          Configuration Method: Automatic
      Proxies:
          Exceptions List: *.local, 169.254/16
          FTP Passive Mode: Yes
      Service Order: 2

    Thunderbolt Bridge:

      Type: Ethernet
      Hardware: Ethernet
      BSD Device Name: bridge0
      IPv4:
          Configuration Method: DHCP
      IPv6:
          Configuration Method: Automatic
      Proxies:
          Exceptions List: *.local, 169.254/16
          FTP Passive Mode: Yes
      Service Order: 3

    Wi-Fi:

      Type: AirPort
      Hardware: AirPort
      BSD Device Name: en0
      IPv4 Addresses: 10.0.0.243
      IPv4:
          Additional Routes:
              Destination Address: 10.0.0.243
              Subnet Mask: 255.255.255.255
              Destination Address: 169.254.0.0
              Subnet Mask: 255.255.0.0
          Addresses: 10.0.0.243
          ARP Resolved Hardware Address: 88:9e:68:f2:d7:9d
          ARP Resolved IP Address: 10.0.0.1
          Configuration Method: DHCP
          Confirmed Interface Name: en0
          Interface Name: en0
          Network Signature: IPv4.Router=10.0.0.1;IPv4.RouterHardwareAddress=88:9e:68:f2:d7:9d
          Router: 10.0.0.1
          Subnet Masks: 255.255.255.0
      IPv6:
          Addresses: 2601:346:b80:4a00:1410:9abd:c19f:347d, 2601:346:b80:4a00:7187:4f9d:35da:84f, 2601:346:b80:4a00::7106, 2601:346:b80:4a00:c8ab:46e4:8a43:dc5b, 2601:346:b80:4a00:e430:7263:18ff:7725, 2601:346:b80:4a00:bd64:2f1c:5880:377f
          Configuration Method: Automatic
          Confirmed Interface Name: en0
          Interface Name: en0
          Network Signature: IPv6.Prefix=2601:346:b80:4a00::/64;IPv6.RouterHardwareAddress=88:9e:68:f2:d7:9d
          Prefix Length: 64, 64, 64, 64, 64, 64
          Router: fe80::8a9e:68ff:fef2:d79d
      DNS:
          Domain Name: hsd1.fl.comcast.net
          Server Addresses: 75.75.75.75, 75.75.76.76, 2001:558:feed::1, 2001:558:feed::2
      DHCP Server Responses:
          Domain Name: hsd1.fl.comcast.net
          Domain Name Servers: 75.75.75.75,75.75.76.76
          Lease Duration (seconds): 0
          DHCP Message Type: 0x05
          Routers: 10.0.0.1
          Server Identifier: 10.0.0.1
          Subnet Mask: 255.255.255.0
      Ethernet:
          MAC Address: 80:a9:97:2b:ed:9d
          Media Options: 
          Media Subtype: Auto Select
      Proxies:
          Exceptions List: *.local, 169.254/16
          FTP Passive Mode: Yes
      Service Order: 4

Describe the bug
After running the project with Docker from the AIO image & installing the dreamshaper model I cannot generate images.

To Reproduce

• Clone repo

• Run docker compose build

• Run docker compose up

• Open web app & install the image generation model

  • Open url in browser http://0.0.0.0:8080/browse/
  • Click image generation
  • Click install for dreamshaper

• Test model/endpoint

curl http://localhost:8080/v1/images/generations \
 -H "Content-Type: application/json" \
 -d '{
    "prompt": "Sketch art style. Black & White. City view from ground level at night|No color. No skyscrapers. No people.",
    "model": "dreamshaper",
    "step": 51,
    "size": "1024x1024"
}'

• Watch API fail.

Expected behavior
I'm expecting it to succeed like the image generation for the "standard" model does...? Apologize if that's not the architecture/name. I'm new to AI/ML. Long story short the one image generation succeeds but the installed model fails.

curl http://localhost:8080/v1/images/generations -H "Content-Type: application/json" -d '{ "prompt": "A stick figure man with a sword",
"size": "256x256"
}'

Logs
The error is that the env doesn't seem to recognize the requirements.txt. I troubleshot it for a while but couldn't get to the bottom of it. I tried to go into the directory ./backend/python/diffusers/backend.py and installing them locally then savings to requirements.txt but this didn't work after rebuilding the image.

api-1  | 10:48PM DBG GRPC Service for DreamShaper_8_pruned.safetensors will be running at: '127.0.0.1:36551'
api-1  | 10:48PM DBG GRPC Service state dir: /tmp/go-processmanager3905376595
api-1  | 10:48PM DBG GRPC Service Started
api-1  | 10:48PM DBG GRPC(DreamShaper_8_pruned.safetensors-127.0.0.1:36551): stdout Initializing libbackend for build
api-1  | 10:48PM DBG GRPC(DreamShaper_8_pruned.safetensors-127.0.0.1:36551): stderr /build/backend/python/diffusers/../common/libbackend.sh: line 78: uv: command not found
api-1  | 10:48PM DBG GRPC(DreamShaper_8_pruned.safetensors-127.0.0.1:36551): stderr /build/backend/python/diffusers/../common/libbackend.sh: line 83: /build/backend/python/diffusers/venv/bin/activate: No such file or directory
api-1  | 10:48PM DBG GRPC(DreamShaper_8_pruned.safetensors-127.0.0.1:36551): stdout virtualenv created
api-1  | 10:48PM DBG GRPC(DreamShaper_8_pruned.safetensors-127.0.0.1:36551): stdout virtualenv activated
api-1  | 10:48PM DBG GRPC(DreamShaper_8_pruned.safetensors-127.0.0.1:36551): stdout activated virtualenv has been ensured
api-1  | 10:48PM DBG GRPC(DreamShaper_8_pruned.safetensors-127.0.0.1:36551): stderr Traceback (most recent call last):
api-1  | 10:48PM DBG GRPC(DreamShaper_8_pruned.safetensors-127.0.0.1:36551): stderr   File "/build/backend/python/diffusers/backend.py", line 13, in <module>
api-1  | 10:48PM DBG GRPC(DreamShaper_8_pruned.safetensors-127.0.0.1:36551): stderr     import torch
api-1  | 10:48PM DBG GRPC(DreamShaper_8_pruned.safetensors-127.0.0.1:36551): stderr ModuleNotFoundError: No module named 'torch'
api-1  | 10:48PM INF Success ip=127.0.0.1 latency="189.542µs" method=GET status=200 url=/readyz
api-1  | 10:48PM ERR failed starting/connecting to the gRPC service error="rpc error: code = Unavailable desc = connection error: desc = \"transport: Error while dialing: dial tcp 127.0.0.1:36551: connect: connection refused\""
api-1  | 10:48PM DBG GRPC Service NOT ready
api-1  | 10:48PM ERR Server error error="grpc service not ready" ip=192.168.65.1 latency=40.133924309s method=POST status=500 url=/v1/images/generations

Additional context
The error is that the env doesn't seem to recognize the requirements.txt. I troubleshot it for a while but couldn't get to the bottom of it. I tried to go into the directory ./backend/python/diffusers/backend.py and installing them locally then savings to requirements.txt but this didn't work after rebuilding the image.

I then tried to run the project locally skipping Docker altogether but ran into the issue mentioned here

Thanks for your help.

[mudler]

@PrimeTimeTran to have support for python-based backend you need to change the IMAGE_TYPE to extras:

LocalAI/docker-compose.yaml

Line 15 in e9c28a1

- IMAGE_TYPE=core

[qdrddr]

I have the same issue. It seems like LocalAI requires a specific version of the protobuf library, that is not installed with Brew. The problem could be with brew not correctly linking the library or with the LocalAI hardcoded version of the library.

But the general problem with macOS & LocalAI is its constantly misaligned library required by localai with the library available on mac, hope that part can be improved in localai.