1. The problem I’m having:
I am trying to create a reverse proxy for famous GenAI tools text-generation-webui and stable diffusion. I believe both of them use Gradio for webui. I configured text gen with the port 8000 for web and 8001 for RestAPI, SD with port 8003 for both web and webui.
Attempt 1:
caddy reverse-proxy --from my.domain.demo --to :8000
Result:
Open my.domain.demo and it works fine although textgen has warning:
Unsupported upgrade request.
No supported WebSocket library detected. Please use "pip install 'uvicorn[standard]'", or install 'websockets' or 'wsproto' manually.
Attempt 2:
my.domain.demo {
log {
output file /var/log/caddy/mydomaincom.log {
roll_size 10mb
roll_keep 20
roll_keep_for 720h
}
}
# Reverse proxy from http://127.0.0.0:8000 to https://my.domain.demo/llm
reverse_proxy /llm localhost:8000 {
header_up Host {host}
header_up Connection {>Connection}
header_up Upgrade {>Upgrade}
header_up X-Real-IP {remote_host}
header_up X-Forwarded-For {remote_host}
header_up X-Forwarded-Proto {scheme}
flush_interval -1
}
# Reverse proxy from http://127.0.0.0:8001 to https://my.domain.demo/llmapi
reverse_proxy /llmapi/* localhost:8001 {
header_up Host {host}
header_up Connection {>Connection}
header_up Upgrade {>Upgrade}
flush_interval -1
}
# Reverse proxy from http://127.0.0.0:8003 to https://my.domain.demo/sd
reverse_proxy /sd/* localhost:8003 {
header_up Host {host}
header_up Connection {>Connection}
header_up Upgrade {>Upgrade}
flush_interval -1
}
# Automatic TLS
tls {
on_demand
}
}
Upload Caddyfile and run commands:
service caddy start
caddy adapt
Save output into caddy.json then:
curl localhost:2019/load -H "Content-Type: application/json" -d @caddy.json
It returns 404/no content.
Also tried adding websocket and handle_path /llm/* but still got same results.
2. Error messages and/or full log output:
https://my.domain.demo/llm and https://my.domain.demo/llmapi
HTTP 404
{
"detail": "Not Found"
}
HTTP 200
No content
3. Caddy version:
v2.7.6 h1:w0NymbG2m9PcvKWsrXO6EEkY9Ru4FJK8uQbYcev1p3A=
4. How I installed and ran Caddy:
a. System environment:
Azure VM
Ubuntu 22.04.3 LTS
b. Command:
apt update
sudo apt install caddy
c. Service/unit/compose file:
none
d. My complete Caddy config:
{"apps":{"http":{"servers":{"srv0":{"listen":[":443"],"logs":{"logger_names":{"my.domain.demo":"log0"}},"routes":[{"handle":[{"handler":"subroute","routes":[{"handle":[{"flush_interval":-1,"handler":"reverse_proxy","headers":{"request":{"set":{"Connection":["{\u003eConnection}"],"Host":["{http.request.host}"],"Upgrade":["{\u003eUpgrade}"],"X-Forwarded-For":["{http.request.remote.host}"],"X-Forwarded-Proto":["{http.request.scheme}"],"X-Real-Ip":["{http.request.remote.host}"]}}},"upstreams":[{"dial":"localhost:8001"}]}],"match":[{"path":["/llmapi/*"]}]},{"handle":[{"flush_interval":-1,"handler":"reverse_proxy","headers":{"request":{"set":{"Connection":["{\u003eConnection}"],"Host":["{http.request.host}"],"Upgrade":["{\u003eUpgrade}"],"X-Forwarded-For":["{http.request.remote.host}"],"X-Forwarded-Proto":["{http.request.scheme}"],"X-Real-Ip":["{http.request.remote.host}"]}}},"upstreams":[{"dial":"localhost:8003"}]}],"match":[{"path":["/sd/*"]}]},{"handle":[{"flush_interval":-1,"handler":"reverse_proxy","headers":{"request":{"set":{"Connection":["{\u003eConnection}"],"Host":["{http.request.host}"],"Upgrade":["{\u003eUpgrade}"],"X-Forwarded-For":["{http.request.remote.host}"],"X-Forwarded-Proto":["{http.request.scheme}"],"X-Real-Ip":["{http.request.remote.host}"]}}},"upstreams":[{"dial":"localhost:8000"}]}],"match":[{"path":["/llm"]}]}]}],"match":[{"host":["my.domain.demo"]}],"terminal":true}]}}},"tls":{"automation":{"policies":[{"on_demand":true,"subjects":["my.domain.demo"]}]}}},"logging":{"logs":{"default":{"exclude":["http.log.access.log0"]},"log0":{"include":["http.log.access.log0"],"writer":{"filename":"/var/log/caddy/mydomaincom.log","output":"file","roll_keep":20,"roll_keep_days":30,"roll_size_mb":10}}}}}