Problem with serving two services over HTTPS on the same host

[Appeltabak (Tonnis Oosterhoff)]

Hi,

I’v got two dockerized services (frontend service and database service) running on the same server. Currently, only the frontend service can be accessed with HTTPS. Here’s its Caddyfile:

subdomain.example.com {
    proxy / grafana:3000 {
        transparant
    }

    gzip

    tls email@example.com
}

That works perfectly. Now I want the connection to the database service on HTTPS too, so I added a section to the Caddyfile:

https://subdomain.example.com:8086 {
    proxy / influxdb:8086 {
        transparant
    }
    gzip
   
    tls email@example.com
 }

As you may have guessed, this fails. I know just a little about web server configuration and HTTPS, so maybe this makes no sense in the first place. My thoughts why this won’t work:

• Host port can’t be the same as the proxy port
• With HTTPS, host names of both services can’t be the same, regardless of different ports.

But I am not sure about this. Further, a limitation applies to my situation:

• Because many other services are already using subdomain.example.com:8086 URL, changing this URL whatsoever to make it work should be considered a last resort, as this would require a lot of work reconfiguring those services.

Long story short: serving two services running on the same server but different ports using HTTPS.

What am I doing wrong? And what’s the way to make it work (if there’s any in this particular situation)?

Thanks

[Whitestrake (Matthew Fay)]

There’s a lot of useful info we’re missing - are you using Docker (with Compose)? What’s Caddy’s log saying when it fails?

@Appeltabak

transparant

Try transparent (spelling).

Receiving a request on port 8086 on the host and proxying to another IP address on the same port shouldn’t be an issue, but it depends on the (Docker?) configuration.

@Appeltabak

many other services are already using subdomain.example.com:8086 URL

Are there other services on the same Caddy host that are trying to use the same URL and port?

[Appeltabak (Tonnis Oosterhoff)]

@Whitestrake

There’s a lot of useful info we’re missing - are you using Docker (with Compose)?

You’re right, I’ve should have been more specific. I’m using Docker 1.12.3 and Compose 1.9.0. Here are the contents of docker-compose.yml:

version: '2.1'

services:
  caddy:
    image: zzrot/alpine-caddy:v0.9.3
    container_name: caddy
    links:
      - grafana
      - influxdb
    ports:
      - "80:80"
      - "443:443"
      - "8086:8086"
    volumes:
      - ./caddy/Caddyfile:/etc/Caddyfile
      - ./caddy:/root/.caddy
    command: caddy -agree -conf /etc/Caddyfile
    logging:
      options:
        max-size: 50m

  influxdb:
    image: influxdb:1.1.0
    container_name: influxdb
    environment:
      - INFLUXDB_REPORTING_DISABLED=true
      - INFLUXDB_ADMIN_ENABLED=false
      - INFLUXDB_HTTP_AUTH_ENABLED=false
    volumes:
      - /app/influxdb:/var/lib/influxdb
    logging:
      options:
        max-size: 50m

  grafana:
    image: grafana/grafana:4.0.1
    container_name: grafana
    links:
      - influxdb:influxdb
    volumes:
      - /app/grafana:/var/lib/grafana
    logging:
      options: 
        max-size: 50m

@Whitestrake

What’s Caddy’s log saying when it fails?

The logs aren’t showing anything specific

caddy       | Activating privacy features... done.
caddy       | http://subdomain.example.com
caddy       | https://subdomain.example.com
caddy       | https://subdomain.example.com:8086

@Whitestrake

Try transparent (spelling).

Oops, forgot to fix that before posting. I already fixed it in the actual Caddyfile, and the problem remains.

@Whitestrake

Are there other services on the same Caddy host that are trying to use the same URL and port?

No, only services running on hosts other than Caddy are using the URL subdomain.example.com:8086

---

I continued investigating, and things became more clear to me. Requests to subdomain.example.com:8086 will fail with ERR_INVALID_HTTP_RESPONSE (using Chrome), but using explicitly https:subdomain.example.com:8086 will succeed. This probably makes sense (as the latter is defined exactly the same in the Caddyfile) , that’s not the behavior I like to see . Any HTTP request on port 8086 should redirect to https:subdomain.example.com:8086

Any thoughts on this?

[Whitestrake (Matthew Fay)]

Docker config looks good.

@Appeltabak

Requests to subdomain.example.com:8086 will fail with ERR_INVALID_HTTP_RESPONSE (using Chrome), but using explicitly https:subdomain.example.com:8086 will succeed. This probably makes sense (as the latter is defined exactly the same in the Caddyfile)

You’re on point with that one - you’ve instructed Caddy to only respond to https requests here. You can remove the https:// from the 8086 vhost and let Caddy’s automatic redirection kick in, or add another block like the following untested example:

http://subdomain.example.com:8086 {
    redir / https://{host}{uri}
}

I believe the {host} placeholder includes port information which will carry through correctly.

[Appeltabak (Tonnis Oosterhoff)]

@Whitestrake

You can remove the https:// from the 8086 vhost and let Caddy’s automatic redirection kick in

Just tested it, but unfortunately it didn’t work. I got the same ERR_INVALID_HTTP_RESPONSE in Chrome. Caddy didn’t log any strange behavior, other than missing a http://subdomain.example.com:8086 entry what I had expected (assuming Caddy’s automatic redirection behavior):

caddy       | Activating privacy features... done.
caddy       | https://subdomain.example.com
caddy       | https://subdomain.example.com:8086
caddy       | http://subdomain.example.com

@Whitestrake

or add another block like the following untested example

Unfortunately, this solution didn’t work either. Caddy exists with the following error:

caddy       | Activating privacy features... done.
caddy       | 2016/12/06 14:06:28 cannot multiplex subdomain.example.com (TLS) and subdomain.example.com (not TLS) on same listener
caddy exited with code 1

I noticed that port is missing from the logs, so maybe Caddy ignores this for some reason? Anyhow, I’m a bit confused now

[Whitestrake (Matthew Fay)]

@Appeltabak

cannot multiplex subdomain.example.com (TLS) and subdomain.example.com (not TLS) on same listener
caddy exited with code 1

Whoops! Damn, that’s me being an idiot, total rookie mistake. Of course you can’t have HTTP(,S) on the same port, same reason standard HTTP is port 80 and HTTPS is port 443. Caddy’s automatic HTTPS wouldn’t have done anything on nonstandard ports anyway, no idea why I suggested it.

In short I think you’re up the creek. Using HTTPS on a nonstandard port you’ll need to specify both scheme and port number for all connecting services, you will not be able to rely on Caddy upgrading HTTP(,S) on the same port.

Assuming your services are redirect friendly anyway, though, you can just open the HTTPS on another port (8087 or 8443 perhaps) and redir from http::8086 → https::8443, which is equivalent to what regular automatic HTTPS does anyway.

[Appeltabak (Tonnis Oosterhoff)]

I’ve been busing with others things, hence the late response

@Whitestrake

Assuming your services are redirect friendly anyway, though, you can just open the HTTPS on another port (8087 or 8443 perhaps) and redir from http::8086 -> https::8443, which is equivalent to what regular automatic HTTPS does anyway.

This seem to work! I have to test thoroughly, though, but I’m positive on this. Thanks for all your time and effort, it has been very helpful!