Switching from nginx to caddy!

[matt (Matt Holt)]

Thanks for submitting the PR. I commented there: godaddy.go provider module... by artknight · Pull Request #58 · libdns/libdns · GitHub

To summarize: what you submitted is the wrapper for Caddy; it’s a package that makes the libdns package usable by Caddy. That’s great, but that belongs in the caddy-dns org on GitHub. But first the libdns package needs to be brought into the libdns org. So either the author will have to do that, or if it’s unmaintained, you’ll have to fork it and maintain it, and submit a PR (just like you did, but with your fork instead) for me to review it.

[artknight (A)]

@francislavoie I just noticed something weird… perhaps you can explain =)

Is there a reason why in this config below requests to honeystalk.io end up inside the *.honeystalk.io?

In other words the request always end up in d:/web/honeystalk.io/account instead of in d:/web/honeystalk.io/site.

honeystalk.io {
    encode gzip

    import ../security-headers.conf
    import ../lucee.conf

    handle_path /shared/* {
        header Access-Control-Allow-Headers "Content-Type"
        header Access-Control-Allow-Origin *
        root * d:/web/honeystalk.io/shared
        file_server
    }

    root * d:/web/honeystalk.io/site

    tls c:/apps/nginx/conf/sites-enabled/certs/honeystalk_io.crt c:/apps/nginx/conf/sites-enabled/certs/honeystalk_io.key
}

*.honeystalk.io {
    encode gzip

    import ../security-headers.conf
    import ../lucee.conf

    handle_path /shared/* {
        header Access-Control-Allow-Headers "Content-Type"
        header Access-Control-Allow-Origin *
        root * d:/web/honeystalk.io/shared
        file_server
    }

    root * d:/web/honeystalk.io/account

    tls c:/apps/nginx/conf/sites-enabled/certs/honeystalk_io.crt c:/apps/nginx/conf/sites-enabled/certs/honeystalk_io.key
}

[francislavoie]

@artknight

In other words the request always end up in d:/web/honeystalk.io/account instead of in d:/web/honeystalk.io/site.

What’s your evidence of that?

[artknight (A)]

Each directory has its own index.cfm file. I added a breakpoint to each and only the one in the .../account gets hit.

[francislavoie]

Please make requests with curl -v that prove this. You can add response headers in each site block to show which one gets hit as well, like header Test apex and header Test sub for example.

[artknight (A)]

Wow, you are still awake

So, I started disabling all the options one by one and this is what causes the weird behavior

I used to have this reverse_proxy http://127.0.0.1:8888

and I changed it yesterday to this

reverse_proxy {
    to http://127.0.0.1:8888
    header_up Host {http.reverse_proxy.upstream.host}
    header_up X-Real-IP {http.reverse-proxy.upstream.address}
}

So, removing the header_up fixed it.

However, I did add the header_up statements b/c I am trying to allow for custom domains to be implemented and according to some docs I needed to do that so that later on in the application I can check the header to know what CNAME was used…if that makes sense

[francislavoie]

Caddy passes through the original Host upstream by default. You don’t need to do anything to preserve it. That header_up line will replace it with 127.0.0.1 which is not what you want here.

Changing Host only makes sense if you need to do it because the upstream makes some decisions based on hostname and it can’t be the original hostname – in which case you’d typically set X-Forwarded-Host as well so no information is lost.

Also, you don’t need X-Real-IP (and that’s the wrong value to set for it anyways) because X-Forwarded-For is already set with the right value.

[artknight (A)]

Thank you for the detailed explanation!

So, to enable custom domains, the user needs to set a CNAME record in their DNS… something like

pm0        CNAME       custom0.honeystalk.io

And the gist of it is … when the user goes to pm0.11bricks.com he/she is sent to custom0.honeystalk.io. However, when I am trying it, I get this

This site can’t provide a secure connection
pm0.11bricks.com sent an invalid response.

When I go to custom0.honeystalk.io everything loads just fine.

Am I missing something?

[francislavoie]

The browser wants a certificate that contains the domain name it made the request for. So that means you need a certificate with pm0.11bricks.com in it.

The way CNAMEs work, is it just makes pm0.11bricks.com use custom0.honeystalk.io to resolve an IP address. The browser doesn’t “see” custom0.honeystalk.io at any point, it just sees the same IP address that that domain happens to resolve to (this is all internal to the DNS resolver).

So for this, you will need Caddy’s On-Demand TLS feature. You can’t use wildcard certs to solve this, because you don’t have control of the customer’s DNS such that you can configure Caddy to change a TXT record automatically (at least I hope you don’t, that would be a too high a level of trust that the customer would have granted you, and that’s dangerous).

Automatic HTTPS - Caddy Documentation

[artknight (A)]

Ok, that’s what I thought! I will be working on getting the godaddy repo forked today and hopefully I can get it submitted correctly this time

[artknight (A)]

On a separate note, do you ever sleep? LOL You are amazing bro!!!

[francislavoie]

My sleep schedule is all over the place. I live alone right now, and we’re still in a pandemic.

[artknight (A)]

How do you test what params/ headers are passed into the ask? Here is the config, let me know if you notice anything.

{
    admin off

    on_demand_tls {
        ask https://honeystalk.io/api/v2/domain-check
        interval 2m
        burst 5
    }

    debug
}

*.honeystalk.io {
    encode gzip

    # log {
    #     output file logs/honeystalk_account_access.log
    #     format json
    # }

    tls {
        on_demand
    }

    import ../security-headers.conf
    import ../lucee.conf

    handle_path /shared/* {
        header Access-Control-Allow-Headers "Content-Type"
        header Access-Control-Allow-Origin *
        root * d:/web/honeystalk.io/shared
        file_server
    }

    root * d:/web/honeystalk.io/account

    import redirected

    tls c:/apps/nginx/conf/sites-enabled/certs/honeystalk_io.crt c:/apps/nginx/conf/sites-enabled/certs/honeystalk_io.key
}

I am assuming there will be a call to https://honeystalk.io/api/v2/domain-check to see if the custom domain is in the DB. I added a catch-all to see what is passed in to that endpoint but not getting it hit for some reason

[matt (Matt Holt)]

@artknight

How do you test what params/ headers are passed into the ask?

It’s basically ?domain= in the URL’s query string with the domain to check. That’s all.

[francislavoie]

Also you probably want to change your site address to https:// instead of *.honeystalk.io, and you’ll need to remove your custom cert line as well.

Setting admin off is not a good idea btw, because the admin endpoint is necessary to perform graceful config reloads of Caddy.

[artknight (A)]

Ok, so here is the main part of the Caddyfile. Please not that requests to honeystalk.io ( no subdomain ) should be routed to a different root than the requests to *.honeystalk.io. How would you re-write it with just https:// {...} wrapper?

honeystalk.io {
    encode gzip

    tls {
        on_demand
    }

    import ../security-headers.conf
    import ../lucee.conf

    handle_path /shared/* {
        header Access-Control-Allow-Headers "Content-Type"
        header Access-Control-Allow-Origin *
        root * d:/web/honeystalk.io/shared
        file_server
    }

    root * d:/web/honeystalk.io/site

    import redirected
}

www.honeystalk.io {
    redir https://honeystalk.io{uri} permanent
}

*.honeystalk.io {
    encode gzip

    tls {
        on_demand
    }

    import ../security-headers.conf
    import ../lucee.conf

    handle_path /shared/* {
        header Access-Control-Allow-Headers "Content-Type"
        header Access-Control-Allow-Origin *
        root * d:/web/honeystalk.io/shared
        file_server
    }

    root * d:/web/honeystalk.io/account

    import redirected
}

[francislavoie]

You don’t need on_demand for domains that are known in your config.

It’s only for domains that are unknown in the config, that Caddy discovers when a client attempts a TLS handshake and provides the domain they want to make a connection for in the SNI field of the handshake.

I think you only actually need 3 site blocks:

honeystalk.io {
	root * d:/web/honeystalk.io/site

	...
}

www.honeystalk.io {
	redir https://honeystalk.io{uri} permanent
}

https:// {
	tls {
		on_demand
	}

	root * d:/web/honeystalk.io/account

	...
}

If you must have a wildcard cert for *.honeystalk.io then that’ll need to be a 4th site block probably just for that, with the appropriate DNS plugin config.

[artknight (A)]

So just to give you some context here, every customer that we have has their own dedicated portal that is essentially a subdomain to honeystalk.io. In that case *.honeystalk.io would work well b/c it would catch all subdomains and serve files from the /account folder.

We also want to offer custom domains to a selected few customers ( that’s is the main reason switching to Caddy…fyi ). So, correct me if i am wrong but I am guessing all four blocks are needed then?

[francislavoie]

@artknight

We also want to offer custom domains to a selected few customers ( that’s is the main reason switching to Caddy…fyi ). So, correct me if i am wrong but I am guessing all four blocks are needed then?

No, you could use On-Demand TLS for your subdomains instead of a wildcard cert (and drop the requirement of building with a DNS plugin altogether), and also handle custom domains the same way with the same site block.

Just make sure the ask endpoint responds with a 200 for valid subdomains as well as custom domains.

[artknight (A)]

When you say drop the requirement of building with a DNS plugin altogether you mean I can achieve having custom domains without the godaddy dns plugin?

If that’s is the case, would you please provide me with the code that would do that