V1 to V2 migration headaches - a troubled start

Help
1

1. Caddy version (caddy version):

v2.1.1 h1:X9k1+ehZPYYrSqBvf/ocUgdLSRIuiNiMo7CvyGUQKeA=

2. How I run Caddy:

a. System environment:

FreeNAS 11.3-U4.1

b. Command:

service caddy start

c. Service/unit/compose file:

#!/bin/sh
#
# $FreeBSD$
#

# PROVIDE: caddy
# REQUIRE: LOGIN DAEMON NETWORKING
# KEYWORD: shutdown

# Add the following lines to /etc/rc.conf.local or /etc/rc.conf
# to enable this service:
# caddy_enable (bool):   Set to NO by default. Set it to YES to enable caddy.
#
# caddy_config (string): Optional full path for caddy config file
# caddy_adapter (string):  Optional adapter type if the configuration is not in caddyfile format
# caddy_extra_flags (string):  Optional flags passed to caddy start
# caddy_logfile (string):     Set to "/var/log/caddy.log" by default.
#                             Defines where the process log file is written, this is not a web access log

. /etc/rc.subr

name=caddy
rcvar=caddy_enable
desc="Caddy 2 is a powerful, enterprise-ready, open source web server with automatic HTTPS written in Go"

load_rc_config $name

# Defaults
: ${caddy_enable:=NO}
: ${caddy_config:="/usr/local/etc/Caddyfile"}
: ${caddy_adapter:=caddyfile}
: ${caddy_extra_flags:=""}
: ${caddy_logfile="/var/log/caddy.log"}

command="/usr/local/bin/${name}"
caddy_flags="--config ${caddy_config} --adapter ${caddy_adapter}"
pidfile="/var/run/${name}.pid"

required_files="${caddy_config} ${command}"

# Extra Commands
extra_commands="validate reload"
start_cmd="${command} start ${caddy_flags} ${caddy_extra_flags} --pidfile ${pidfile} >> ${caddy_logfile} 2>&1"
validate_cmd="${command} validate ${caddy_flags}"
reload_cmd="${command} reload ${caddy_flags}"

run_rc_command "$1"

d. My complete Caddyfile or JSON config:

{
  acme_ca https://acme-staging-v02.api.letsencrypt.org/directory
  email stu.pitt@dumas.com
}

www.udance.com.au udance.com.au {
  encode gzip
  log {
    output file /var/log/udance.com.au.log
    format console
  }

  reverse_proxy https://10.1.1.23 {
    transport http {
      tls_insecure_skip_verify
    }
  }
}

3. The problem I’m having:

A stripped-down, but working V1 Caddyfile:

www.udance.com.au udance.com.au {
  gzip
  proxy / http://10.1.1.23 {
    transparent
  }
}

When I try (what I think is) the V2 version of the Caddyfile (with some staging and extra logging), and try to access the site, I first see:

screenshot.390
Clicking Advanced and proceeding, I see…

screenshot.389

4. Error messages and/or full log output:

The console log…

{"level":"info","ts":1596223924.742527,"msg":"using provided configuration","config_file":"/usr/local/www/Caddyfile","config_adapter":"caddyfile"}
{"level":"info","ts":1596223924.7474186,"logger":"admin","msg":"admin endpoint started","address":"tcp/localhost:2019","enforce_origin":false,"origins":["localhost:2019","[::1]:2019","127.0.0.1:2019"]}
2020/08/01 03:32:04 [INFO][cache:0xc0006507e0] Started certificate maintenance routine
{"level":"info","ts":1596223924.7482967,"logger":"http","msg":"server is listening only on the HTTPS port but has no TLS connection policies; adding one to enable TLS","server_name":"srv0","https_port":443}
{"level":"info","ts":1596223924.7483904,"logger":"http","msg":"enabling automatic HTTP->HTTPS redirects","server_name":"srv0"}
{"level":"info","ts":1596223924.7534752,"logger":"tls","msg":"cleaned up storage units"}
{"level":"info","ts":1596223924.7536786,"logger":"http","msg":"enabling automatic TLS certificate management","domains":["www.udance.com.au","udance.com.au"]}
{"level":"info","ts":1596223924.7878313,"msg":"autosaved config","file":"/.config/caddy/autosave.json"}
{"level":"info","ts":1596223924.7878964,"msg":"serving initial configuration"}
Successfully started Caddy (pid=34086) - Caddy is running in the background
2020/08/01 03:32:18 http: TLS handshake error from 10.1.1.149:52847: remote error: tls: unknown certificate
2020/08/01 03:32:18 http: TLS handshake error from 10.1.1.149:52852: remote error: tls: unknown certificate
2020/08/01 03:32:21 http: TLS handshake error from 10.1.1.149:52862: remote error: tls: unknown certificate
{"level":"error","ts":1596223941.691412,"logger":"http.log.error.log0","msg":"dial tcp 10.1.1.23:443: connect: connection refused","request":{"method":"GET","uri":"/","proto":"HTTP/2.0","remote_addr":"10.1.1.149:52864","host":"www.udance.com.au","headers":{"Cache-Control":["max-age=0"],"Upgrade-Insecure-Requests":["1"],"Accept":["text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9"],"Sec-Fetch-Site":["none"],"Sec-Fetch-User":["?1"],"Sec-Fetch-Dest":["document"],"Accept-Encoding":["gzip, deflate, br"],"User-Agent":["Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4147.105 Safari/537.36 Edg/84.0.522.49"],"Sec-Fetch-Mode":["navigate"],"Accept-Language":["en-US,en;q=0.9"]},"tls":{"resumed":false,"version":772,"ciphersuite":4865,"proto":"h2","proto_mutual":true,"server_name":"www.udance.com.au"}},"duration":0.001372649,"status":502,"err_id":"80v91b7zx","err_trace":"reverseproxy.(*Handler).ServeHTTP (reverseproxy.go:411)"}
{"level":"info","ts":1596223949.649602,"msg":"shutting down apps then terminating","signal":"SIGTERM"}
2020/08/01 03:32:29 [INFO][cache:0xc0006507e0] Stopped certificate maintenance routine
{"level":"info","ts":1596223949.6507356,"logger":"admin","msg":"stopped previous server"}
{"level":"info","ts":1596223949.6508896,"msg":"shutdown done","signal":"SIGTERM"}

The access log…

1.5962239416915581e+09  error   http.log.access.log0    handled request {"request": {"method": "GET", "uri": "/", "proto": "HTTP/2.0", "remote_addr": "10.1.1.149:52864", "host": "www.udance.com.au", "headers": {"Cache-Control": ["max-age=0"], "Upgrade-Insecure-Requests": ["1"], "Accept": ["text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9"], "Sec-Fetch-Site": ["none"], "Sec-Fetch-User": ["?1"], "Sec-Fetch-Dest": ["document"], "Accept-Encoding": ["gzip, deflate, br"], "User-Agent": ["Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/84.0.4147.105 Safari/537.36 Edg/84.0.522.49"], "Sec-Fetch-Mode": ["navigate"], "Accept-Language": ["en-US,en;q=0.9"]}, "tls": {"resumed": false, "version": 772, "ciphersuite": 4865, "proto": "h2", "proto_mutual": true, "server_name": "www.udance.com.au"}}, "common_log": "10.1.1.149 - - [01/Aug/2020:03:32:21 +0800] \"GET / HTTP/2.0\" 502 0", "duration": 0.001372649, "size": 0, "status": 502, "resp_headers": {"Server": ["Caddy"]}}

5. What I already tried:

I’ve tried twiddling as many knobs as I can, but have got nowhere. There’s something in the logs, I’ve tried searching the forum using various keywords from the logs, but I’m completely bamboozled. Looking at the logs reminds of what Cypher said to Neo in one of my favourite movies ‘There’s way too much information to decode the Matrix’.

6. Links to relevant resources:

N/A

2

Well the obvious difference is that you’re using https:// for your proxy in v2 but http:// in v1. Why the difference?

This is the important bit from the logs:

2 Likes
3

@francislavoie Thanks for the rapid response.

That’s interesting, because if I use http:// instead, Caddy doesn’t start and the log shows:

{"level":"info","ts":1596225351.4866734,"msg":"using provided configuration","config_file":"/usr/local/www/Caddyfile","config_adapter":"caddyfile"}
run: adapting config using caddyfile: parsing caddyfile tokens for 'reverse_proxy': /usr/local/www/Caddyfile:17 - Error during parsing: upstream address scheme is HTTP but transport is configured for HTTP+TLS (HTTPS)
start: caddy process exited with error: exit status 1
4

What’s curious is I thought I had the reverse proxy, using the https:// form, working a couple of months back when I was doing some initial testing with V2.0.0. At the time I used the pre-built binary. It’s strange that it’s not working any longer. The difference this time around is I’m using V2.1.1 and building from xcaddy.

5

That’s because you specified http:// but then went on to configure tls_insecure_skip_verify which is for https://, so the parser is confused as to which it should use:

upstream address scheme is HTTP but transport is configured for HTTP+TLS (HTTPS)

2 Likes
6

Are you trying to tell me it’s as simple as this?!

www.udance.com.au udance.com.au {
  encode gzip
  reverse_proxy http://10.1.1.23
}

I tried it. It works! I LOVE CADDY V2 :heart_eyes:

Thank you, gentlemen!

3 Likes
7

Yep, minus that errant { at the top of your config :stuck_out_tongue:

Glad you got it :smile:

1 Like
8

Gotcha! I got too excited and didn’t check my work.

1 Like
9

Some additional comments that might be useful to other readers and a related question:

The Caddyfile code block defined above (and repeated below) works for TLS with HTTP validation:

www.udance.com.au udance.com.au {
  encode gzip
  reverse_proxy http://10.1.1.23
}

When Caddy is built with a DNS provider, say Cloudflare, the Caddyfile code block below works for TLS with DNS validation:

www.udance.com.au udance.com.au {
  encode gzip
  reverse_proxy http://10.1.1.23
  tls {
    dns cloudflare *APItoken*
  }  
}

The follow-up question I have is:

When Caddy is built for DNS validation, does the TLS directive need to be defined per sub-domain (i.e. per Caddyfile code block), or can it be defined once globally?

10

Currently, per server block. There’s no way to do it globally yet. But you can use snippets to avoid repetition.

1 Like
11

That’s useful to know. Snippets works for me.

(dnschallenge) {
  tls {
    dns cloudflare *APItoken*
  }
}

www.udance.com.au udance.com.au {
  encode gzip
  reverse_proxy http://10.1.1.23
  import dnschallenge
}

Cool! Only the snippet needs to be changed if you change DNS provider, or roll the API token.

1 Like
12

This topic was automatically closed after 30 days. New replies are no longer allowed.