Letsencrypt rate limiting at 1 per 10 minutes?

Richard_Stupek · 2018-05-15 19:04:17 UTC

I’m trying to figure out why the following is in the code

mholt/caddy/blob/master/caddytls/handshake.go#L333


// Copyright 2015 Light Code Labs, LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
//     http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.


package caddytls


import (
	"crypto/tls"
	"fmt"
	"strings"


	"github.com/mholt/caddy/telemetry"
)


// configGroup is a type that keys configs by their hostname
// (hostnames can have wildcard characters; use the getConfig
// method to get a config by matching its hostname).
type configGroup map[string]*Config


// getConfig gets the config by the first key match for name.
// In other words, "sub.foo.bar" will get the config for "*.foo.bar"
// if that is the closest match. If no match is found, the first
// (random) config will be loaded, which will defer any TLS alerts
// to the certificate validation (this may or may not be ideal;
// let's talk about it if this becomes problematic).
//
// This function follows nearly the same logic to lookup
// a hostname as the getCertificate function uses.
func (cg configGroup) getConfig(name string) *Config {
	name = strings.ToLower(name)


	// exact match? great, let's use it
	if config, ok := cg[name]; ok {
		return config
	}


	// try replacing labels in the name with wildcards until we get a match
	labels := strings.Split(name, ".")
	for i := range labels {
		labels[i] = "*"
		candidate := strings.Join(labels, ".")
		if config, ok := cg[candidate]; ok {
			return config
		}
	}


	// try a config that serves all names (the above
	// loop doesn't try empty string; for hosts defined
	// with only a port, for instance, like ":443") -
	// also known as the default config
	if config, ok := cg[""]; ok {
		return config
	}


	return nil
}


// GetConfigForClient gets a TLS configuration satisfying clientHello.
// In getting the configuration, it abides the rules and settings
// defined in the Config that matches clientHello.ServerName. If no
// tls.Config is set on the matching Config, a nil value is returned.
//
// This method is safe for use as a tls.Config.GetConfigForClient callback.
func (cg configGroup) GetConfigForClient(clientHello *tls.ClientHelloInfo) (*tls.Config, error) {
	config := cg.getConfig(clientHello.ServerName)
	if config != nil {
		return config.tlsConfig, nil
	}
	return nil, nil
}


// ClientHelloInfo is our own version of the standard lib's
// tls.ClientHelloInfo. As of May 2018, any fields populated
// by the Go standard library are not guaranteed to have their
// values in the original order as on the wire.
type ClientHelloInfo struct {
	Version            uint16        `json:"version,omitempty"`
	CipherSuites       []uint16      `json:"cipher_suites,omitempty"`
	Extensions         []uint16      `json:"extensions,omitempty"`
	CompressionMethods []byte        `json:"compression,omitempty"`
	Curves             []tls.CurveID `json:"curves,omitempty"`
	Points             []uint8       `json:"points,omitempty"`


	// Whether a couple of fields are unknown; if not, the key will encode
	// differently to reflect that, as opposed to being known empty values.
	// (some fields may be unknown depending on what package is being used;
	// i.e. the Go standard lib doesn't expose some things)
	// (very important to NOT encode these to JSON)
	ExtensionsUnknown         bool `json:"-"`
	CompressionMethodsUnknown bool `json:"-"`
}


// Key returns a standardized string form of the data in info,
// useful for identifying duplicates.
func (info ClientHelloInfo) Key() string {
	extensions, compressionMethods := "?", "?"
	if !info.ExtensionsUnknown {
		extensions = fmt.Sprintf("%x", info.Extensions)
	}
	if !info.CompressionMethodsUnknown {
		compressionMethods = fmt.Sprintf("%x", info.CompressionMethods)
	}
	return telemetry.FastHash([]byte(fmt.Sprintf("%x-%x-%s-%s-%x-%x",
		info.Version, info.CipherSuites, extensions,
		compressionMethods, info.Curves, info.Points)))
}


// ClientHelloTelemetry determines whether to report
// TLS ClientHellos to telemetry. Disable if doing
// it from a different package.
var ClientHelloTelemetry = true

It seems that you’re rate limiting to one certificate issued every 10 minutes after the first 10?

matt · 2018-05-15 19:36:26 UTC

Yep, the rate limits for on-demand TLS are documented here: https://caddyserver.com/docs/automatic-https#on-demand

Richard_Stupek · 2018-05-15 19:40:02 UTC

Ok. Why one every 10 minutes after the first 10? There isn’t any way to configure it right?

matt · 2018-05-15 19:58:47 UTC

To prevent abuse. And no, it can’t be configured. But you can use the ask subdirective to do your own abuse prevention.

oliverdunk · 2018-05-20 12:45:01 UTC

Hey @matt ! This feature is really exciting for a project that I’m working on. I want to enable HTTPS for the custom domains added by users to my SaaS app.

I was just wondering if I am ok to tweak these rate limits by modifying the source, or if that is a very bad idea? In particular:

Remove the hard limit on the number of certificates.
After the first ten, allow one certificate to be requested each minute, rather than one every ten.

The above would be following these assumptions:

There are no failed validations, or they are at least rare, because my ask endpoint will only return a 200 if the DNS is setup correctly.
The number of requests by a given user or for a given domain name is extremely limited by the ask endpoint, say five domain changes per user per week.

I understand why the rate limits are in place in the source and am not suggesting that these are changed for those using Caddy out of the box. I was just wondering if changing them for our own usage is ok or if we should be looking at making a custom Caddyfile loader instead?

Whitestrake · 2018-05-20 13:44:24 UTC

Just my 2c… There’s two questions there - can you, and should you?

Firstly, yeah, sure - the code is all Apache licensed, you’re allowed to modify it for your usage.

Secondly, it’s there to protect both LetsEncrypt’s endpoint from abuse, and Caddy’s users from rate limits - so as long as you’re aware of the responsibility you’re taking into your own hands - that is, to be fair to LE’s ACME server, and to avoid being rate-limited - the choice is ultimately yours.

oliverdunk · 2018-05-20 13:48:08 UTC

Thanks @Whitestrake, that makes sense.

If you don’t mind me rephrasing the question: would you, wanting to achieve the feature I am suggesting, do what I am proposing?

My main reason for asking here is because I appreciate the limit is there for a reason and am wondering if there is a better way. Based on what you have said I think I am making the right decision here but I would appreciate any feedback if you disagree

Whitestrake · 2018-05-20 13:50:57 UTC

Yeah, probably. I don’t think there are two ways about it; if you need more certs than the current cap, you need to remove the cap, and the above code is where you do it. The technical part is pretty straight-forward.

The rate limits are all published, so if you make sure the endpoint you’re using for ask validation keeps those in mind, you’ll have no problems.

matt · 2018-05-20 14:54:20 UTC

I would just set up an ask endpoint that does the rate limiting or whatever other checks you want, then you can use stock binaries/source code. Those limits you proposed sound fine; it ultimately depends on your threat model – just having some limits in place is a huge net positive, so that’s good.