Load_folders , tls_connection_policies for multiple domain

Suman_Chandel1 · December 11, 2020, 7:12am

1. Caddy version (`caddy version`):

v2.2.1

2. How I run Caddy:

caddy run -config caddy.json

a. System environment:

Mac OS

d. My complete Caddyfile or JSON config:

{
	"apps": {
		"http": {
			"servers": {
				"example": {
					"listen": [":443"],
					"routes": [
						{
							"match": [
                                {
                                    "host": [
                                        "api.edjx.test"
                                    ]
                                },
                                {
                                    "path": [
                                        "*"
                                    ]
                                }
                            ],
							"handle": [{
								"handler": "static_response",
								"body": "Hello, .. world!\n"
							}]
                        },
                        {
							"match": [
                                {
                                    "host": [
                                        "fn.edjx.test"
                                    ]
                                },
                                {
                                    "path": [
                                        "*"
                                    ]
                                }
                            ],
							"handle": [{
								"handler": "static_response",
								"body": "Hello, fn world!\n"
							}]
						}
                    ],
                    "tls_connection_policies": [
                        {
                          "match": {
                            "sni": ["api.edjx.test"]
                          },
                          "certificate_selection": {
                            "all_tags": ["api"]
                          }
                        },
                        {
                          "match": {
                            "sni": ["fn.edjx.test"]
                          },
                          "certificate_selection": {
                            "all_tags": ["fn"]
                          }
                        }
                      ]
				}
			}
		},
		"tls": {
            "certificates": {
                "load_folders": ["/edjx/caddy_test/", "/edjx/caddy_test/fn/"]
            }
        }
	},
    "logging": {
      "logs": {
        "default": {
          "level": "DEBUG"
        }
      }
    }
}

3. The problem I’m having:

Problem is that how certificates will be picked for multiple domains when load_folders are used. Its working fine when i use it for single domain , and picks the certificate directly, but when i use two domains, then what should be values for tls_connection_policies ?

4. Error messages and/or full log output:

Not picking the certificates,
if i remove tls_connection_policies ,then response are not coming , every time it is picking the api.edjx.test and returns Hello … world

5. What I already tried:

6. Links to relevant resources:

Suman_Chandel1 · December 14, 2020, 4:27am

There are few questions that i have related with Scalability…

We are thinking to use caddy as gateway for thousand of rules and hundreds in domain at our platform. Customers will be hosting their domain at our network.Could you help in sharing the numbers how much caddy is scalable in terms of rules and domains? If any benchmarking page is available, if link can be shared it would be more helpful.
Related with above ticket, we have four methods, automate, load_pem , load_folders , load_files which method is recommended in which use case and which one is more scalable ?

francislavoie · December 14, 2020, 7:09am

Sorry, I won’t be of much help about your original question with certificates, hopefully @matt will be able to answer when he has the time.

Caddy is known to be used by companies with thousands of domains, so your scale shouldn’t be a problem. Caddy can be scaled out horizontally as well; if they all share the same storage backend, then Caddy can act in a cluster by using the storage backend as a distributed lock for certificate management.

I strongly suggest you get a support contract if you plan to use Caddy at such a scale. That way you can get prioritized help.

https://caddyserver.com/business

matt · December 14, 2020, 4:54pm

The template wasn’t completely filled out.

The problem I’m having:
Problem is that how certificates will be picked for multiple domains when load_folders are used. Its working fine when i use it for single domain , and picks the certificate directly, but when i use two domains, then what should be values for tls_connection_policies ?

Error messages and/or full log output:
Not picking the certificates,
if i remove tls_connection_policies ,then response are not coming , every time it is picking the api.edjx.test and returns Hello … world

Can you be more specific? Use curl -v to show requests that are working and requests that aren’t, also post the log output? There’s not enough information here, I’m not quite sure what you’re asking.

There are few questions that i have related with Scalability…

We are thinking to use caddy as gateway for thousand of rules and hundreds in domain at our platform. Customers will be hosting their domain at our network.Could you help in sharing the numbers how much caddy is scalable in terms of rules and domains? If any benchmarking page is available, if link can be shared it would be more helpful.

As long as your hardware can support it, you should be good for up to hundreds of thousands of domains. I’m pretty confident because there are already single Caddy instances serving tens of thousands on low-end hardware.

Make sure you monitor logs for certificate errors. If too many automated/managed certificates fail repeatedly (we’re talking hundreds I think), they will starve other domains from getting certificates. The alternative is depleting available resources, which IMO a server should never do. So don’t let too many errors build up.

Related with above ticket, we have four methods, automate, load_pem , load_folders , load_files which method is recommended in which use case and which one is more scalable ?

Why not just let Caddy manage the certificates for you?

Keep in mind that loading them manually has a high overhead at config load time because there’s lots of decoding that has to happen. When Caddy manages certs for you, you can enable on-demand TLS which defers cert loading and maintenance to request-time, as needed. So, in other words, that’s what the automate loader does (but you still have to enable on-demand TLS separately in your config). The other loaders expect that you’re manually managing certificates, which is not recommended.

Suman_Chandel1 · December 15, 2020, 9:12am

Hi @matt,

Thanks for responding!

I was able to resolve this with different configs for multiple config. Below is working Config

{
    "apps": {
        "http": {
            "servers": {
                "example": {
                    "listen": [
                        ":443"
                    ],
                    "routes": [
                        {
                            "handle": [
                                {
                                    "handler": "static_response",
                                    "body": "Hello, api test world!\n"
                                }
                            ],
                            "match": [
                                {
                                    "host": [
                                        "api.edjx.test"
                                    ]
                                }
                            ],
                            "terminal": true
                        },
                        {
                            "handle": [
                                {
                                    "handler": "static_response",
                                    "body": "Hello, fn test world!\n"
                                }
                            ],
                            "match": [
                                {
                                    "host": [
                                        "fn.edjx.test"
                                    ]
                                }
                            ],
                            "terminal": true
                        }
                    ]
                }
            }
        },
        "tls": {
            "certificates": {
                "load_folders": [
                    "/Users/sumanchandel/workspace/edjx/caddy_test/"
                ]
            }
        }
    },
    "logging": {
        "logs": {
            "default": {
                "level": "DEBUG"
            }
        }
    }
}

But I still have a few questions related to certificate management via automate method.

Based on your recommendation, the automate method is the best for thousands of domains. Could we manage the certificates for the domains that we don’t own, If yes, could you share the details or some link/example?
Could you provide the caddy benchmarking link?

matt · December 15, 2020, 2:28pm

Based on my limited understanding of what you want, your config above looks good.

To be clear, you don’t need to specify the automate loader for any domains that are in host matchers in top-level HTTP routes. Automatic HTTPS will automate certificates for those domains for you (unless you disable it, of course). See the documentation:

However, as in your case, note that one of the ways to disable certificate management for a domain is to “manually load certificates” for it:

Manually loading certificates (unless this config property is true)

So if your folder has a certificate for api.edjx.test, for example, Caddy will not manage another certificate for that domain. It will simply use that manually-loaded certificate instead. Be sure to renew it before it expires and reload the config. In this way, manual certificates and automated certificates can be used in tandem, if that’s what you need.

So unless you want to manually manage certificates for thousands of domains, I would let Caddy handle that instead. That’s… kind of the point.

Yes, but be aware that the domains still have to be configured properly: namely, their A/AAAA records need to be pointed to your Caddy instance and ideally, both HTTP requests and TLS connections need to get through without termination.

What you want is a feature called On-Demand TLS, one which to my knowledge is exclusive to Caddy. It triggers certificate management at handshake-time. All the docs you need are linked to from here:

This allows your customers to configure their domains whenever they get around to it, and when Caddy starts seeing connections to it, Caddy will begin managing certificates for them.

Be sure to configure the “ask” endpoint to prevent abuse.

What is that?

system · January 10, 2021, 7:12am

This topic was automatically closed after 30 days. New replies are no longer allowed.