Use my own domain purely internally?

Help
1

Hey guys,

there is one simple thing I want to do: Cleaning up my internal subdomains.

Some background info:

  • I use pfSense as my router and DHCP server.
  • piHole acts as my DNS server.
  • The only thing coming into my network is a single VPN connection.
  • I use a domain internally (I specifically bought it for this purpose and use it for nothing else). There are no A or MX records configured for it.

How it works right now:
I have all the different services on one subnet. Each service gets its own subdomain (apart from plex) via its hostname. So I can get to pihole via “pihole.domain.tld”

What I don’t like about that:
Some services require a specific port and or directory to access them. Proxmox for example is available under 10.10.10.30:8006… → This is not good as I have to remember all those special snowflakes…
Additionally most services produce errors, when loading the site, because the site doesn’t use any encryption… I had my own self-signed certificate once, but more services were installed and it then was too much work to self-sign a certificate for every service…

How I plan to do change it:
Get a reverse proxy with built-in letsencrypt support. I already tried nginx and traefik, but that was too complex for me… I just couldn’t figure out how to port that documentation-knowledge to my specific use case…

So I laid my eyes upon Caddy. It looks really simple and lightweight!
I would set up my DNS server to redirect all services to caddy. Because I already have the hostnames in place, I came up with a system: the hostname is the name of the service, but for “subdomain-use” I would specify a new “hostname” which essentially gets the description of the service.
One Example: instead of typing “pihole.domain.tld”, I will type “adblocker.domain.tld”. Exactly that new subdomain (“adblocker.halnet.cloud”) will be inserted into my DNS server to redirect to Caddy. Caddy will then know what to do with it and redirect it to the real IP and slap some encryption on there. (Yeah I know only the path between caddy and browser is then encrypted, but I still get the green lock! And it only is my home network after all)

What I have tried and what didnt work:
I installed caddy correctly and it works. Then I added a Caddyfile with just my “domain.tld” in it. I then ran caddy and it asked for my email. I used the same email which I also used for buying that domain. Then it threw me an error that there is no IP connected to that IP or something… I tried pointing my domain to my router (which I didn’t really want to do), but still nothing… It then threw errors, that there is a different ip connected to it… sadly I can’t remember and now I can’t even see those errors anymore, because it just says: Error creating new authz :: too many failed authorizations recently

So what can I do to use my domain purely internally?

2

Personally I use the DNS challenge for getting certificates. This means that I don’t need to forward any traffic from outside to my internal caddy. This sounds similar to what you want to achieve so might be worth checking if the provider you use for DNS is supported.

3

yes I use godaddy!

I also had something working for one service, but then I changed it to work with all services and that must have borked something bi time…

could you post/send me your config or the relevant part of your config, so that I can get an idea?

4

Your Router + Cloudflare + DynV6

  1. Router:
  • set your DMZ or
  • port triggering and/or mapping
  • open port 80 and 443 (out) is needed
  1. Cloudflare:
  • Go to DNS > CNAME
  • name: sub.yourdomain.tld
  • value: yourname.dynv6.net
  1. DynV6:
  • create your account
  • setting up and connect to ipv6.he.net
    • follow the DynV6 step-by-step @ instructions tab
  • make sure that .sh file (Linux, Mac or Windows) is running with crontab or similar service

Done.

I’m running Caddy and Canonical Landscape here on baremetal.

Important note: If you need more than one subdomain with different IPv6 , you need more than one ethernet card. Each ethernet card gives one different IPv6 and, @ DynV6 + HE.NET you can setting up one more canonical address. In the end, you can a different service for Caddy.

5

Thank you for your reply! But that is not really what I want…I dont want any domain pointing at my router. I just want to use that domain in LAN. The services inside my LAN won’t go outside and nothing will go in (except for the VPN).

Meanwhile I hacked something together:

pihole dns entries:
address=/halnet.cloud/10.10.10.32
address=/router.halnet.cloud/10.10.10.32
address=/pdu.halnet.cloud/10.10.10.32
address=/jarvis.halnet.cloud/10.10.10.32
address=/deepblue.halnet.cloud/10.10.10.32
address=/deepthought.halnet.cloud/10.10.10.32
address=/switch.halnet.cloud/10.10.10.32
address=/switchx.halnet.cloud/10.10.10.32
address=/switchd.halnet.cloud/10.10.10.32
address=/virt.halnet.cloud/10.10.10.32
address=/dns.halnet.cloud/10.10.10.32
address=/proxy.halnet.cloud/10.10.10.32
address=/wifi.halnet.cloud/10.10.10.32
address=/remote.halnet.cloud/10.10.10.32
address=/wiki.halnet.cloud/10.10.10.32
address=/git.halnet.cloud/10.10.10.32
address=/money.halnet.cloud/10.10.10.32
address=/ticket.halnet.cloud/10.10.10.32
address=/genisys.halnet.cloud/10.10.10.32
address=/icarus.halnet.cloud/10.10.10.32
address=/nas.halnet.cloud/10.10.10.32
address=/media.halnet.cloud/10.10.10.32
address=/backup.halnet.cloud/10.10.10.32

Next I created my caddyfile, whic you can see down below.

then installed caddy: sudo curl https://getcaddy.com | bash -s personal http.forwardproxy,net,tls.dns.godaddy

and finally the following:

GODADDY_API_KEY=hfbhjsrgnjskjgbbgkgb GODADDY_API_SECRET=jrcungvqtnwengqvnn caddy

but I ran into an issue: I reached one of let’sEncrypt’s rate limits. I don’t know which exact limit though… So I gradually reduced the number of proxys in my caddyfile and finally got something working. Here and there some services still want to do http, but I guess this will be solved with some variables… Proxmox won’t allow me to connect, so there is still an issue at hand as well.

My biggest problem now though: I can’t get certs for all services! I have read about wildcards, so there must be something more elegant than this “medusa of config”… I just don’t know what to do…

Caddyfile

  tls {
    dns godaddy
  }
  proxy / 10.10.10.1
}

pdu.halnet.cloud {
  tls {
    dns godaddy
  }
  proxy / 10.10.10.2
}

jarvis.halnet.cloud {
  tls {
    dns godaddy
  }
  proxy / 10.10.10.3
}

deepblue.halnet.cloud {
  tls {
    dns godaddy
  }
  proxy / 10.10.10.4
}

deepthought.halnet.cloud {
  tls {
    dns godaddy
  }
  proxy / 10.10.10.6
}

switch.halnet.cloud {
  tls {
    dns godaddy
  }
  proxy / 10.10.10.8
}

switchx.halnet.cloud {
  tls {
    dns godaddy
  }
  proxy / 10.10.10.9
}

switchd.halnet.cloud {
  tls {
    dns godaddy
  }
  proxy / 10.10.10.10
}

halnet.cloud {
  tls {
    dns godaddy
  }
  proxy / 10.10.10.21
}

virt.halnet.cloud {
  tls {
    dns godaddy
  }
  proxy / 10.10.10.30:8006
}

dns.halnet.cloud {
  tls {
    dns godaddy
  }
  proxy / 10.10.10.31/admin
}

proxy.halnet.cloud {
  tls {
    dns godaddy
  }
  proxy / 10.10.10.35/books/halnetcloud/page/proxy
}

wifi.halnet.cloud {
  tls {
    dns godaddy
  }
  proxy / 10.10.10.33:8080
}

remote.halnet.cloud {
  tls {
    dns godaddy
  }
  proxy / 10.10.10.34:8080/guacamole
}

wiki.halnet.cloud {
  tls {
    dns godaddy
  }
  proxy / 10.10.10.35
}

git.halnet.cloud {
  tls {
    dns godaddy
  }
  proxy / 10.10.10.36
}

money.halnet.cloud {
  tls {
    dns godaddy
  }
  proxy / 10.10.10.37
}

ticket.halnet.cloud {
  tls {
    dns godaddy
  }
  proxy / 10.10.10.38
}

genisys.halnet.cloud {
  tls {
    dns godaddy
  }
  proxy / 10.10.10.51
}

icarus.halnet.cloud {
  tls {
    dns godaddy
  }
  proxy / 10.10.10.52
}

nas.halnet.cloud {
  tls {
    dns godaddy
  }
  proxy / 10.10.10.90:80
}

media.halnet.cloud {
  tls {
    dns godaddy
  }
  proxy / 10.10.10.90:32400
}

backup.halnet.cloud {
  tls {
    dns godaddy
  }
  proxy / 10.10.10.100
}
6

Oh. Ok. I am sorry. I am so sorry.
Well, a Caddy instance (systemd) as DNS Server (CoreDNS like) can help, no?

I mean, for setting up custom domains, tld and more.

A solution perhaps:

7

no worries, even though I kind of know what I want to do, I still can’t really explain it… so the problem really is on my side here! :smiley:

I had already figured out DNS a while ago with pihole (which took me some time as well). So pihole works for me now, no real need to change it. Once you know what file to edit, it is dead simple.

I could be happy just using pihole and a certbot or something, but pihole as just a dns server won’t allow urls with ports or directories. So that solution leaves out my problem that I dont want to remember all those pesky ports and special directories…

EDIT
and thank you for that acme.sh! This might be a solution, if I went away from caddy! This script is probably the best base to then build my own script ontop of!

8

Hi @TheGammelgalopper,

No wildcards in Caddy yet, unfortunately :frowning:

Last I heard these were still delayed by LetsEncrypt. This is a good thing, we definitely want to see them get it right rather than fast.

You’ve already posted a link to the rate limit page - my guess is you’ve hit the Certificates per Registered Domain limit (20/wk). You’ve got over 20 subdomains of halnet.cloud just in your Caddyfile. You’ll have to wait a week to collect 'em all; this isn’t a Caddy issue, per se, in that going to another LE script won’t solve the issue for this domain.

I’mn surprised that PiHole doesn’t let you set *.halnet.cloud to 10.10.10.32 with a single set of records. I use DNSBL on my pfSense (using the same blocking ruleset as the standard PiHole), and in my Unbound custom configuration, I use

local-zone: "example.com" redirect
local-data: "example.com 86400 IN A 192.168.2.10"

in order to split DNS locally to my web server; any request in the example.com zone (including all child, grandchild, etc subdomains) gets sent to that IP.

1 Like
9

But the wildcard is working fine on LE already? I don’t know about Certbot.

10

Hardly “working fine on LE already”. There is a testing endpoint; your own link points to this forum post, which, as of March 5, indicates that testing is still underway with no release date.

Additionally, we depend on xenolf/lego for our ACME functionality, and we’ll need to change how we handle wildcards in the Caddyfile, so I imagine there will be a bit of delay from the public release and a release of Caddy with the new functionality included.

11

Oh, yes. I mean, fine when using ACMEv2 testing, I guess, but do not on Caddy.
The first test, as far as I remember, it was in 27/02 or release date for that.

Oh, ok. So, we need three steps until release a Caddy with this functionality. The LetsEncrypt official release, xenolf/lego and, lastly, the Caddy. Considering wildcard release by LetsEncrypt, how many time as usual till the Caddy release?

Anyway, acme.sh works on it also.

12

Caddy doesn’t have a release cadence - versions come out as features are completed and determined to be ready for release. But at a guess, since wildcards are a hotly anticipated feature and some work has already been completed towards supporting them (see https://github.com/mholt/caddy/pull/2015), it will be done pretty quick.

1 Like
13

Hi @Whitestrake ,

thank you for your comprehensive comment! This has really helped me a lot! I guess one more person is now waiting for wildcard certs :wink:

Yes that sounds reasonable. I didnt know how to do this otherwise. Is there really no other way to simplify my caddyfile until wildcard certs come out?
So I will have to wait a week before actually being able to use this? Or is there a way to run caddy although it gives this error? (without making the caddyfile smaller for the first week though)
Would you recommend any other highly important settings in my caddyfile?

Why didn’t I think about that?? Yeah sure pihole uses dnsmasq, sure it can do that! It works perfectly! Now at least my DNS setting is a lot simpler… thx!

Do you have any tips on troubleshooting the services which now don’t work with https?
pfSense runs fine… proxmox for example gives me a 502 bad gateway… but this isn’t too urgent! I will probably troubleshoot these errors in the upcoming weeks…

14

Combine Caddy with acme.sh? I didn’t try yet. acme.sh works on wildcard already. Check the link above, please.

15

But Let’s Encrypt does not yet issue trusted wildcard certificates. Just to be sure that is clear.

2 Likes
16

@perroud yeah sure I could add acme.sh and add an extra level of complexity or I just wait until it comes out with Caddy… :smiley: I already spent quite some time figuring out what I have now (4-5 hours). There is this one basic question that I have to ask myself now: Will I spend many more hours hacking more stuff together or do I wait until it all works the simple way… :wink:
Still there’s quite some stuff do adjust (for example proxmox, dell idrac and many more…). It just doesn’t make any sense at the moment to throw another variable in the box.

If I turned my back to the Caddy project, I would probably look at acme.sh. But for the foreseeable future I see Caddy as my best option!

I guess I am lazy now and want to go the safe way. @matt and whoever else is doing the heavy lifting here will make us all happy. I am very sure about that! :sunglasses::hugs:

17

Nice. If you want to try, something like that:

acme.sh  --issue --test  \
    --dns dns_dp \
    -d halnet.cloud \
    -d *.halnet.cloud \
    --keylength ec-256

:vulcan_salute:

1 Like
18

Best I can think of is to try and put any back-end services that support setting a URL Base on the same hostname and in subfolders instead, that would at least get you online.

Unfortunately, yes, you will have to wait a week to get more than 20 certificates out of LetsEncrypt in this circumstance. Probably the single biggest detractor from LetsEncrypt - regardless of which client you use, you’ll hit that rate limit if you’ve got lots of subdomains on the same domain name.

http://www.dot.tk provide free domain names, you could grab one or two from there in the interim and spread your services out over that to get everything up and running with HTTPS.

You can run HTTP-only, or you can run TLS On-Demand (which I’d discourage as it will hammer LetsEncrypt whenever you make requests, and then fail anyway). But there is no way to force Caddy to run despite certificate errors - you’re effectively telling it to start and manage your HTTPS for you, and it can’t do that, so it has to fail out.

Hmm. Gzip and logging maybe? Sounds like a perfect use case for snippets, actually:

(standard) {
  gzip
  log /var/log/caddy/access.log
  errors /var/log/caddy/errors.log
  tls {
    godaddy
  }
}

# In each site definition:
example.halnet.cloud {
  proxy / 10.10.10.xxx
  import standard
}

Might also want to check which back-ends would benefit from the transparent proxy preset.

It looks like all your proxies go to HTTP upstreams, and generally speaking, the first step to troubleshooting these is always to curl that upstream from the Caddy host, because that’s pretty much what Caddy itself is doing when you make these requests.

The errors directive might also shed some light (timeouts, hostname didn’t resolve, etc).

1 Like
19

Oh yeah subfolders might be an idea! Why didnt I think about that… :roll_eyes: I have three servers with management cards. So I could do something like manage.halnet.cloud/server1-3! That’s great!

I guess I will wait then… But I will also redo my Caddyfile with subfolders where it is possible and resonable!

Ok that makes total sense! Of course a program fails, when it can’t do one of its main purposes… Sorry for not thinking that through thoroughly…

Thank you for the optimizaton recommendations! I will read into it and integrate where needed!

But what did I get myself into here? This topic is far deeper than I had expected (although I am probably just scratching the surface here… :D).

Thank you all for your help (Especially @Whitestrake of course)! This was truly awesome feedback!

1 Like
20

I could get some services to work! I combined @Whitestrake 's suggestions with some of my own research. Of course the current solution is far from perfect and I am still trying to get a working config for iDRAC, Firefly III and Bookstack. For the last two I will probably open issues in their respective github issues (as they somehow don’t respond to certian curls…)

But these here work:

Config for unifi controller:

wifi.halnet.cloud {
proxy / https://10.10.10.33:8443 {
insecure_skip_verify
transparent
}
import standard
}

Config for proxmox:

virt.halnet.cloud {
proxy / https://10.10.10.30:8006 {
insecure_skip_verify
}
import standard
}

Config for gitlab:

git.halnet.cloud {
proxy / 10.10.10.36 {
transparent
}
import standard
}

EDIT
note to myself: make sure to disable gzip for Guacamole Server. It absolutely kills it, prob too high latency…