Docker +Caddy+ Cloudflare +Docker Proxy

1. Caddy version (caddy version):

Caddy 2.2.1

2. How I run Caddy:

Docker Container

a. System environment:

OMV5
Debian Buster
Docker
Portainer

c. Service/unit/compose file:

Docker

d. My complete Caddyfile or JSON config:

    requests.movies4you.digital {
        encode gzip
        log {
        output file /var/log/ombi_access.log {
            roll_size 1gb
            roll_keep 5
            roll_keep_for 720h    
        }
    }
        reverse_proxy /* localhost:3579 {
        }
        tls email here {
            dns cloudflare API-KEY
        }
    }
    accounts.movies4you.digital {
        encode gzip
        log {
            output file /var/log/accounts_access.log {
                roll_size 50mb
                roll_keep 5
                roll_keep_for 720h                
        }
    }
        reverse_proxy /* localhost:8056 {         
        }
        tls email here {
            dns cloudflare API-KEY
        }
    }
    stream.movies4you.digital {
        encode gzip
        log {
            output file /var/log/jf_access.log {
                roll_size 50mb
                roll_keep 5
                roll_keep_for 720h   
        }
    }
        reverse_proxy /* localhost:8096 {         
        }
        tls email here {
            dns cloudflare API-KEY
        }
    }
        lidarr.movies4you.digital {
        encode gzip
        log {
            output file /var/log/lidarr_access.log {
                roll_size 50mb
                roll_keep 5
                roll_keep_for 720h             
        }
    }
        reverse_proxy /* localhost:8686 {         
        }
        tls email here {
            dns cloudflare API-KEY
        }
    }

        sonarr.movies4you.digital {
        encode gzip
        log {
            output file /var/log/sonarr_access.log {
                roll_size 50mb
                roll_keep 5
                roll_keep_for 720h             
        }
    }
        reverse_proxy /* localhost:8989 {         
        }
        tls email here {
            dns cloudflare API-KEY
        }
    }
        radarr.movies4you.digital {
        encode gzip
        log {
            output file /var/log/radarr_access.log {
                roll_size 50mb
                roll_keep 5
                roll_keep_for 720h             
        }
    }
        reverse_proxy /* localhost:7878 {         
        }
        tls email here email here {
            dns cloudflare API-KEY
        }
    }
        comics.movies4you.digital {
        encode gzip
        log {
            output file /var/log/radarr_access.log {
                roll_size 50mb
                roll_keep 5
                roll_keep_for 720h             
        }
    }
        reverse_proxy /* localhost:8090 {         
        }
        tls email here {
            dns cloudflare API-KEY
        }
    }

3. The problem I’m having:

I actually have a couple of questions. Whats the difference in using a docker proxy with cloudflare here: Docker Hub compared to just using a docker hub with a cloudflare plugin built on docker hubs abiosoft caddy rep here: Docker Hub
does the docker proxy with cloudflare allow me access to all the containers where the docker hub caddy with cloudflare allow me to not access those containers?
Also could someone poiny me in the general direction for a good setup for docker + cloudflare + caddy2+ docker proxy (if needed)
Can someone possibly give me an in depth tutorial. I couldnt find much either than what was on Docker Hub

4. Error messages and/or full log output:

Cant get it to run the certs through. No 80 and 443 are both open as well as firewall on Router and OS are correctly opened. My ISP is also not blocking the ports. I cant get a cert to process. It loads but wont process. Im assuming because of outdated docker hub dockerfiles. Thank you ahead of time.

5. What I already tried:

I’ve have already tried countless different docker hub dockerfiles and setups. I only got one to write a cert but it didnt work with cloudflare. Soo…

6. Links to relevant resources:

https://registry.hub.docker.com/r/abiosoft/caddy
https://hub.docker.com/r/sabrsorensen/caddy-docker-proxy-cloudflare

2 Likes

The sabrsorensen/caddy-docker-proxy-cloudflare image is just Caddy + GitHub - lucaslorentz/caddy-docker-proxy: Caddy as a reverse proxy for Docker + GitHub - caddy-dns/cloudflare: Caddy module: dns.providers.cloudflare.

You can build that yourself very easily with the official Caddy builder image from Docker Hub (see the section titled “Adding custom Caddy modules”). The above is just someone’s convenience docker image they published, it’s nothing official.

The abiosoft/caddy is Caddy v1 only, so you should no longer be using that. Please use the official caddy image instead, i.e. Docker Hub

The lucaslorentz/caddy-docker-proxy plugin is essentially an alternate mode of configuring Caddy, by using docker labels to generate a Caddyfile config on the fly. It’s super cool, but you’ll need to weigh if that makes sense for you. It’s generally best used if you have a setup where you spin up new containers you want to proxy to on a frequent basis. If your stack is relatively static and non-changing, then this probably isn’t worth setting up for you.

The caddy-dns/cloudflare plugin is what provides support for the ACME DNS challenge with Cloudflare. So if you have Cloudflare as your DNS provider, you’ll need that.

Your Caddyfile looks like a bit of a mess, so I recommend running the caddy fmt command on it to clean up the indentation, it’ll make it much easier to read.

You seem to be using localhost:<port> for all your proxies. If you’re running in Docker, this is likely not correct – localhost refers to the same container as Caddy, so if those services are in other containers, Caddy won’t try to reach them. Instead, you should be using the container name of those services.

And finally, you generally have your proxy lines set up like this:

reverse_proxy /* localhost:8096 {         
}

The /* is unnecessary and very slightly slower than just omitting it, because you’re telling Caddy to use a path matcher that will always match, so it needs to do a path comparison, where-as if you just omit it, it won’t need to do a comparison. This’ll be on the order of nanoseconds, but it reads cleaner in the config anyways. Also, the { } braces are unnecessary, because you have nothing within them. So do this instead:

reverse_proxy <the container name>:8096
1 Like

OMFFGGG! I LOVE YOU IVE BEEN TRYING TO GET THIS WORKING FOR MONTHS. I never knew that the reverse_proxy jellyfin:8096 would work. Ive looked high and low for how to access caddy through docker and access outside the docker container. I couldnt find squat. I think you guys should make this more known. I also used this docker hub which has cloudflare already implemented if anyone has trouble building docker pull mrkaran/caddy:latest . But again fircking rock. Thank you for the tips as well i hvae already implemented everythng that was said. I apologise for the late response. I was working on a little bit of a different project for a few days,

2 Likes

quick question i got the reverse proxy working but its not loading my docker container when it loads theres also is no cloudflare error, Any ideas? It does succesfully create the certs i can see that in hte console and logs

:sweat_smile: glad you got it working!

There’s plenty of documentation throughout the site. I think it’s well documented.

I’m not sure I understand the question, could you please elaborate? Please be specific. What do you mean by “not loading”?

Honestly, I strongly recommend you don’t use a random docker image you find on docker hub, there isn’t the same level of trust there. They might later push a image version that does something you don’t expect. Instead, write your own Dockerfile like I pointed to in the Docker Hub docs:

FROM caddy:2.2.1-builder AS builder

RUN xcaddy build \
    --with github.com/caddy-dns/cloudflare

FROM caddy:2.2.1

COPY --from=builder /usr/bin/caddy /usr/bin/caddy

okay i will think about implementing that. I mean its not loading the webpage. i type in stream.movies4you.digital and nothing loads on that domain not even a 4** error from cloudflare any ideas? The local 192.168.1.116:8096 works but the domain wont load with the reverse proxy only the internal ip will load. i have set the dns external ip in cloudflare to an A record with that and the hostname movies4you.digital with an orange proxy. the rest of the subdomains ie CNAME records are set to DNS and page rules is set to bypass for cache. So i dont have to buy cloudflare premium or breaks the Terms and Conditons/go over on data.

What’s your Caddyfile? What’s in your logs? What do you see if you run curl -v to make the request instead of your browser?

caddyfile

requests.movies4you.digital {
    encode gzip
    log {
    output file /var/log/ombi_access.log {
        roll_size 1gb
        roll_keep 5
        roll_keep_for 720h    
    }
}
    reverse_proxy /ombi:3579
    tls {
        dns cloudflare api key
    }
}
accounts.movies4you.digital {
    encode gzip
    log {
        output file /var/log/accounts_access.log {
            roll_size 50mb
            roll_keep 5
            roll_keep_for 720h                
    }
}
    reverse_proxy /* jfa-go:8056
    tls {
        dns cloudflare api key
    }
}
stream.movies4you.digital {
    encode gzip
    log {
        output file /var/log/jf_access.log {
            roll_size 50mb
            roll_keep 5
            roll_keep_for 720h   
    }
}
    reverse_proxy jellyfin:8096
    tls {
        dns cloudflare api key
    }
}
    lidarr.movies4you.digital {
    encode gzip
    log {
        output file /var/log/lidarr_access.log {
            roll_size 50mb
            roll_keep 5
            roll_keep_for 720h             
    }
}
    reverse_proxy /* lidarr:8686
    tls {
        dns cloudflare api key
    }
}

    sonarr.movies4you.digital {
    encode gzip
    log {
        output file /var/log/sonarr_access.log {
            roll_size 50mb
            roll_keep 5
            roll_keep_for 720h             
    }
}
    reverse_proxy sonarr:8989
    tls {
        dns cloudflare api key
    }
}
    radarr.movies4you.digital {
    encode gzip
    log {
        output file /var/log/radarr_access.log {
            roll_size 50mb
            roll_keep 5
            roll_keep_for 720h             
    }
}
    reverse_proxy radarr:7878 {         
    }
    tls {
        dns cloudflare api key
    }
}
    comics.movies4you.digital {
    encode gzip
    log {
        output file /var/log/mylar3_access.log {
            roll_size 50mb
            roll_keep 5
            roll_keep_for 720h             
    }
}
    reverse_proxy /* mylar3:8090
    tls {
        dns cloudflare api key
    }
}
    movies4you.digital {
    encode gzip
    log {
        output file /var/log/homer_access.log {
            roll_size 50mb
            roll_keep 5
            roll_keep_for 720h             
    }
}
    reverse_proxy homer:6666
    tls {
        dns cloudflare api key
    }
}

the logs look good.

{"level":"info","ts":1604094908.497121,"logger":"tls.obtain","msg":"releasing lock","identifier":"requests.movies4you.digital"},
{"level":"info","ts":1604094917.3809137,"logger":"tls.obtain","msg":"certificate obtained successfully","identifier":"radarr.movies4you.digital"},
{"level":"info","ts":1604094917.3809283,"logger":"tls.obtain","msg":"releasing lock","identifier":"radarr.movies4you.digital"},
{"level":"info","ts":1604094916.6176395,"logger":"tls.issuance.acme.acme_client","msg":"validations succeeded; finalizing order","order":"https://acme-v02.api.letsencrypt.org/acme/order/100851369/5955098008"},
{"level":"info","ts":1604094916.795844,"logger":"tls.issuance.acme.acme_client","msg":"validations succeeded; finalizing order","order":"https://acme-v02.api.letsencrypt.org/acme/order/100851368/5955097997"},
{"level":"info","ts":1604094916.7977045,"logger":"tls.issuance.acme.acme_client","msg":"validations succeeded; finalizing order","order":"https://acme-v02.api.letsencrypt.org/acme/order/100851367/5955097994"},
{"level":"info","ts":1604094917.1970406,"logger":"tls.issuance.acme.acme_client","msg":"successfully downloaded available certificate chains","count":2,"first_url":"https://acme-v02.api.letsencrypt.org/acme/cert/03bdfeff930f2aa217fbfe2a68d52532bb1d"},
{"level":"info","ts":1604094916.7031827,"logger":"tls.issuance.acme.acme_client","msg":"validations succeeded; finalizing order","order":"https://acme-v02.api.letsencrypt.org/acme/order/100851371/5955097982"},
{"level":"info","ts":1604094917.1974783,"logger":"tls.obtain","msg":"releasing lock","identifier":"lidarr.movies4you.digital"},
{"level":"info","ts":1604094917.1974583,"logger":"tls.obtain","msg":"certificate obtained successfully","identifier":"lidarr.movies4you.digital"},
{"level":"info","ts":1604094917.2491758,"logger":"tls.issuance.acme.acme_client","msg":"successfully downloaded available certificate chains","count":2,"first_url":"https://acme-v02.api.letsencrypt.org/acme/cert/045a854b1d406b30a8695df784402fa7b9b6"},
{"level":"info","ts":1604094917.2496455,"logger":"tls.obtain","msg":"releasing lock","identifier":"comics.movies4you.digital"},
{"level":"info","ts":1604094917.318841,"logger":"tls.issuance.acme.acme_client","msg":"successfully downloaded available certificate chains","count":2,"first_url":"https://acme-v02.api.letsencrypt.org/acme/cert/0470fb8ad9f2a37de53d9701319c40c56bdd"},
{"level":"info","ts":1604094917.2495873,"logger":"tls.obtain","msg":"certificate obtained successfully","identifier":"comics.movies4you.digital"},
{"level":"info","ts":1604094917.3193944,"logger":"tls.obtain","msg":"releasing lock","identifier":"sonarr.movies4you.digital"},
{"level":"info","ts":1604094917.3805654,"logger":"tls.issuance.acme.acme_client","msg":"successfully downloaded available certificate chains","count":2,"first_url":"https://acme-v02.api.letsencrypt.org/acme/cert/04332dcb4623db6cd59cd0be4b2070d02248"},
{"level":"info","ts":1604094908.497089,"logger":"tls.obtain","msg":"certificate obtained successfully","identifier":"requests.movies4you.digital"},
{"level":"info","ts":1604094907.9700549,"logger":"tls.issuance.acme.acme_client","msg":"validations succeeded; finalizing order","order":"https://acme-v02.api.letsencrypt.org/acme/order/100851365/5955097905"},
{"level":"info","ts":1604094908.4964705,"logger":"tls.issuance.acme.acme_client","msg":"successfully downloaded available certificate chains","count":2,"first_url":"https://acme-v02.api.letsencrypt.org/acme/cert/0345df15330c95e234099348017561fd5348"},
{"level":"info","ts":1604094917.3193767,"logger":"tls.obtain","msg":"certificate obtained successfully","identifier":"sonarr.movies4you.digital"},
{"level":"info","ts":1604094907.5352592,"logger":"tls.obtain","msg":"certificate obtained successfully","identifier":"stream.movies4you.digital"},
{"level":"info","ts":1604094907.5352874,"logger":"tls.obtain","msg":"releasing lock","identifier":"stream.movies4you.digital"}

what am i curl -v the domain or the ip adress and is it like pinging an ip?

edit: curl -v stream.movies4you.digitla
response:

  • Expire in 13 ms for 1 (transfer 0x5635d3762dc0)
  • Trying (myexternalip)…
  • TCP_NODELAY set
  • Expire in 200 ms for 4 (transfer 0x5635d3762dc0)
  • connect to (my external ip) port 80 failed: Connection refused
  • Failed to connect to stream.movies4you.digital port 80: Connection refused
  • Closing connection 0
    curl: (7) Failed to connect to stream.movies4you.digital port 80: Connection refused

Please use the caddy fmt command to clean up your Caddyfile syntax, it’s hard to follow because the indentation is not correct.

reverse_proxy /ombi:3579

Don’t put / there. It’ll try to make a request to a hostname /ombi which is not a thing.

reverse_proxy /* jfa-go:8056

Same here, remove /* as I explained earlier.

curl makes an HTTP request, so you use the same address as you would in a browser.

ping is not HTTP, it’s just raw TCP packets with a command asking to have the message sent back. Read about it here: Communication Networks/Ping - Wikibooks, open books for an open world

This means that you weren’t able to make a connection to your Caddy server. Did you expose ports 80 and 443 to the internet? That domain resolves to this IP for me:

$ host stream.movies4you.digital
stream.movies4you.digital has address 69.246.201.79

You might want to read this:

Also, FYI you could avoid a lot of the repetition in your Caddyfile by using snippets.

2 Likes

the caddy concepts i would implement but cloudflare automatically redirects to https. the import (Caddyfile directive) — Caddy Documentation i have never heard of but the cloudflare automatically also propegates the Origin certs and implements them. i made the changes to the caddyfile. i apologize i wasnt paying attention i made some edits to see if something would work. then reverted back and didnt realize i sent the wrong file to you. This is my current caddy:

requests.movies4you.digital {
    encode gzip
    log {
    output file /var/log/ombi_access.log {
        roll_size 1gb
        roll_keep 5
        roll_keep_for 720h    
    }
}
    reverse_proxy ombi:3579
    tls {
        dns cloudflare api key
    }
}
accounts.movies4you.digital {
    encode gzip
    log {
        output file /var/log/accounts_access.log {
            roll_size 50mb
            roll_keep 5
            roll_keep_for 720h                
    }
}
    reverse_proxy jfa-go:8056
    tls {
        dns cloudflare api key
    }
}
stream.movies4you.digital {
    encode gzip
    log {
        output file /var/log/jf_access.log {
            roll_size 50mb
            roll_keep 5
            roll_keep_for 720h   
    }
}
    reverse_proxy jellyfin:8096
    tls {
        dns cloudflare api key
    }
}
    lidarr.movies4you.digital {
    encode gzip
    log {
        output file /var/log/lidarr_access.log {
            roll_size 50mb
            roll_keep 5
            roll_keep_for 720h             
    }
}
    reverse_proxy lidarr:8686
    tls {
        dns cloudflare api key
    }
}

    sonarr.movies4you.digital {
    encode gzip
    log {
        output file /var/log/sonarr_access.log {
            roll_size 50mb
            roll_keep 5
            roll_keep_for 720h             
    }
}
    reverse_proxy sonarr:8989
    tls {
        dns cloudflare api key
    }
}
    radarr.movies4you.digital {
    encode gzip
    log {
        output file /var/log/radarr_access.log {
            roll_size 50mb
            roll_keep 5
            roll_keep_for 720h             
    }
}
    reverse_proxy radarr:7878 {         
    }
    tls {
        dns cloudflare api key
    }
}
    comics.movies4you.digital {
    encode gzip
    log {
        output file /var/log/mylar3_access.log {
            roll_size 50mb
            roll_keep 5
            roll_keep_for 720h             
    }
}
    reverse_proxy mylar3:8090
    tls {
        dns cloudflare api key
    }
}
    movies4you.digital {
    encode gzip
    log {
        output file /var/log/homer_access.log {
            roll_size 50mb
            roll_keep 5
            roll_keep_for 720h             
    }
}
    reverse_proxy homer:6666
    tls {
        dns cloudflare api key
    }
}

ive opened port 80 i guess i though port 88 was 80. it looked like it was 80 but ive open it now but the stream.movies4you.digital still will not load. I tested the curl -v stream.movies4you.digital

response:

  • Trying 69.246.201.79…
  • TCP_NODELAY set
  • Expire in 200 ms for 4 (transfer 0x555676268dc0)
  • Connected to stream.movies4you.digital (69.246.201.79) port 80 (#0)

GET / HTTP/1.1
Host: stream.movies4you.digital
User-Agent: curl/7.64.0
Accept: /

< HTTP/1.1 308 Permanent Redirect
< Connection: close
< Location: https://stream.movies4you.digital/
< Server: Caddy
< Date: Sat, 31 Oct 2020 01:57:31 GMT
< Content-Length: 0
<

  • Closing connection 0

At this point it’s not an issue with Caddy but rather with either your Cloudflare setup or your networking setup. Caddy does get reached on port 80 but it seems something is preventing the connection on port 443 (HTTPS).

okay thank you for the help any ideas on how to troubleshoot the cloudflare issues. btw i have omv5 installed which runs docker and portainer could that have something to do with it

do i need to add any code when starting a container in docker. I have all ports being shown as closed. ive checked my router their port forwarded and i am 1000% sure caddy is running correctly. But do i have to add any code tot he dockercompose file to make it run through caddy?

Please see Docker Hub there’s a docker-compose example there.

ive already seen that documentation. I was talking about the networks in portainer/docker do i need to link it in networks or do i need to add extra docker commands to mount other containers in docker to use and see docker caddy container. Also the certs are succesfully created but port 80 and 443 are showing closed on my external facing ip. Ive checked my router and have succesfully open 443 and 80. I have no idea why the ports are showing not opened but my caddy container succesfully created the certs on those ports. Any ideas. This physically has me stumped.

Sorry, I don’t know anything about portainer.

Make sure your router is port forwarding ports 80 and 443 to the machine running your Caddy. Otherwise the router won’t know to which machine to send requests on those ports to.

If you can go on your phone, use cellular data (not wifi) and make a request to your site, then you’re good to go. Some sites that check for “ports being open” might not be checking for the right signals. It’s usually a best-effort guess from those tools.

its a docker issue. i need to make a netwok and attach the containers with the caddy network bridge. I did it in the past but cant figure out or find how i did it? Im 1000% sure i portfwarded.
EDIT: Im using a tor onion configured on my router with fresh tomato firmware on Netgear R7000. Will this effect the portforword or caddy.

anyone have any insite. I cant figure out why my domain is not showing externally. Ive tried checking the ports, ive tried checking my router, no firewall blocks, and some how im still not able to get even a cloudflare error. Its just a blank page. im assuming something to do with linking caddy and the container in dockers network. Could anyone be of assistance.

okay now im sure its a caddy configuration issue. when i install through docker its not creating a custom network like others could someone help me fix this issue. my containers are not being exposed to the outside world for that exact reason.

This topic was automatically closed after 30 days. New replies are no longer allowed.