Reverse-proxy using docker-caddy-jenkins

Help
1

1. Caddy version (caddy version):

v2 latest docker version

2. How I run Caddy:

in compose file: caddy run --config “/etc/caddy/Caddyfile” --adapter “caddyfile”

a. System environment:

NAME=“Oracle Linux Server”
VERSION=“8.2”

Client: Docker Engine - Community
Version: 19.03.13
API version: 1.40

b. Command:

sudo -E docker-compose up

c. Service/unit/compose file:

version: "3"
services:
  caddy:
    image: caddy
    container_name: caddy-container
    #user: 
    #privileged: 
    restart: unless-stopped    
    #environment:    
    command: caddy run --config "/etc/caddy/Caddyfile" --adapter "caddyfile"
    expose:
      - "80"
      - "443"
    ports:
      - "80:80"
      - "443:443"
    volumes: 
        #when using sudo, you get https://forums.docker.com/t/docker-compose-not-seeing-environment-variables-on-the-host/11837/7
        #So use the -E when starting docker-compose
      - ${PWD}/caddy/data:/data
      - ${PWD}/caddy/Caddyfile:/etc/caddy/Caddyfile
      - ${PWD}/caddy/config:/config
      - ${PWD}/caddy/www:/usr/share/caddy
    #depends_on:
    #networks:

  jenkins:
    image: jenkinsci/blueocean #uses LTS
    container_name: jenkins-container
    user: root
    privileged: true
    restart: unless-stopped    
    environment:
      #- JAVA_OPTS="-Xmx4096m"
      #- JENKINS_HOME=${PWD}/jenkins-data
      - DOCKER_HOST=tcp://docker:2376
      - DOCKER_CERT_PATH=/certs/client
      - DOCKER_TLS_VERIFY=1
    #command:
    expose:
      - "8080"
      - "50000"
    ports:
      - "8080:8080"
      - "50000:50000"
    volumes:
      - ${PWD}/jenkins-data:/var/jenkins_home
      - ${PWD}/jenkins-docker-certs:/certs/client:ro
    #depends_on:
    #networks:
    
  dind:
    image: docker:dind
    container_name: docker-dind-container
    #user: 
    privileged: true
    restart: unless-stopped
    environment:
      - DOCKER_TLS_CERTDIR=/certs
    #command:
    #expose:
    ports:
      - "2376:2376"
    volumes:
      - ${PWD}/jenkins-data:/var/jenkins_home
      - ${PWD}/jenkins-docker-certs:/certs/client
    #depends_on:
    #networks:

d. My complete Caddyfile or JSON config:

{
    # email to use on Let's Encrypt
    email @ domain.com

    #acme_ca https://acme-staging-v02.api.letsencrypt.org/directory
    #debug
}
#1
#jenkins.domain.com {
  #respond "Service currently unavailable"
  #reverse_proxy jenkins-container:8080
#}

#2
#jenkins.domain.com {
#    reverse_proxy {
#        to http://jenkins-container:8080
#        health_path /health
#        health_status 200
#        transport http {
#            read_buffer 4096
#        }
#    }
#}

3. The problem I’m having:

I’m trying to protect jenkins-container:8080 http://jenkins.domain.com:8080 by reverse-proxy it through caddy https://jenkins.domain.com. But after the setup the internal address is yet accessible to external world.

4. Error messages and/or full log output:

The first setup (#1) is working, but everyone can hit http://jenkins.domain.com:8080 - shouldn’t it be blocked by default?
The second one (#2) does not work, when I try the https://jenkins.domain.com it give me a blank page and the following error:

jenkins-container | 2020-10-14 17:09:09.664+0000 [id=28]        INFO    jenkins.InitReactorRunner$1#onAttained: Completed initialization
jenkins-container | 2020-10-14 17:09:09.756+0000 [id=20]        INFO    hudson.WebAppMain$3#run: Jenkins is fully up and running
caddy-container | {"level":"info","ts":1602695368.9761386,"logger":"http.handlers.reverse_proxy.health_checker.active","msg":"unexpected status code","status_code":403,"host":"jenkins-container:8080"}
caddy-container | {"level":"error","ts":1602695378.3320687,"logger":"http.log.error","msg":"no upstreams available","request":{"remote_addr":"10.9.1.111:41566","proto":"HTTP/2.0","method":"GET","host":"jenkins.domain.com","uri":"/","headers":{"User-Agent":["Mozilla/5.0 (X11; Linux x86_64; rv:78.0) Gecko/20100101 Firefox/78.0"],"Accept":["text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8"],"Accept-Language":["pt-BR,pt;q=0.8,en-US;q=0.5,en;q=0.3"],"Accept-Encoding":["gzip, deflate, br"],"Cookie":["_ga=GA1.4.832148051.1581363323; jenkins-timestamper-offset=10800000; screenResolution=1920x1080; JSESSIONID.f7f59b99=node0bwtd7djb4a2ri32b5vmxm0sb20.node0; JSESSIONID.a654ec24=node01it9hlj8laneb16x3sd34sztq20.node0"],"Upgrade-Insecure-Requests":["1"],"Te":["trailers"]},"tls":{"resumed":false,"version":772,"cipher_suite":4865,"proto":"h2","proto_mutual":true,"server_name":"jenkins.domain.com"}},"duration":0.000057614,"status":502,"err_id":"78yy7z56a","err_trace":"reverseproxy.(*Handler).ServeHTTP (reverseproxy.go:440)"}
caddy-container | {"level":"info","ts":1602695398.9765975,"logger":"http.handlers.reverse_proxy.health_checker.active","msg":"unexpected status code","status_code":403,"host":"jenkins-container:8080"}
caddy-container | {"level":"info","ts":1602695428.9754827,"logger":"http.handlers.reverse_proxy.health_checker.active","msg":"unexpected status code","status_code":403,"host":"jenkins-container:8080"}
caddy-container | {"level":"info","ts":1602695458.9752088,"logger":"http.handlers.reverse_proxy.health_checker.active","msg":"unexpected status code","status_code":403,"host":"jenkins-container:8080"}

I can access (http://jenkins.domain.com:8080) normally.

5. What I already tried:

6. Links to relevant resources:

2

That’s because you’re binding port 8080 to your host machine. Don’t do that. Remove these lines from your docker-compose.yml:

    ports:
      - "8080:8080"
      - "50000:50000"

For your second issue, the result is clear:

caddy-container | {"level":"info","ts":1602695368.9761386,"logger":"http.handlers.reverse_proxy.health_checker.active","msg":"unexpected status code","status_code":403,"host":"jenkins-container:8080"}

You told Caddy to expect status 200 from jenkins on the path /health but jenkins responded with status 403. You’ll need to figure out the right way to set up health checks.

Apparently you can use the login page as a signal that the service is up: Jenkins Health Check URL - DevOps Stack Exchange

Or you could just omit health checks, because they aren’t that useful when you’re only proxying to one backend. Health checks are more useful when you’re load balancing between multiple instances, so that Caddy can find out when one goes down, to take it out of rotation.

2 Likes
3

Thanks, Francis! Commenting the ports section

   ports:
      - "8080:8080"
      - "50000:50000"

and removing

 health_path /health
 health_status 200

make it work.
One more thing, in the jenkins docs(Reverse proxy configuration) there is one section about reverse-proxy and several adjustment for many different reverse proxies. Do you think Caddy has this need?

Again, thank you!

1 Like
4

Probably not. The nginx example is the most similar to Caddy, but most of it is already covered automatically by Caddy. The only thing is that it’s set up to have nginx serve /userContent, but that’s likely just a minor optimization, not necessary.

1 Like
5

This topic was automatically closed after 30 days. New replies are no longer allowed.