Caddy + OPNSense Internal working, external not

Help
1

1. The problem I’m having:

Running OPNSENSE OPNsense 24.1.5_2-amd64 with Caddy Plugin os-caddy 1.5.3
Also using a custom certificate via os-acme-client 4.1
DNS is provided by Porkbun

connecting to a service on the local LAN (service1.mydomain.com) resolves and connects properly
connecting to a service from the wider Internet (service1.mydomain.com) fails silently and timesout

2. Error messages and/or full log output:

Connections from external requests do not show up in the logs.

3. Caddy version:

Running OPNSENSE OPNsense 24.1.5_2-amd64 with Caddy Plugin os-caddy 1.5.3

4. How I installed and ran Caddy:

Installed plugin via the OPNSENSE firmware repository via the GUI.

a. System environment:

Running OPNSENSE OPNsense 24.1.5_2-amd64

b. Command:

No Commands, all config done via GUI

c. Service/unit/compose file:

Not using Docker

d. My complete Caddy config:

# DO NOT EDIT THIS FILE -- OPNsense auto-generated file


# Global Options
{
	log {
		output net unixgram//var/caddy/var/run/log {
		}
		format json {
			time_format rfc3339
		}
	}

	dynamic_dns {
		provider porkbun {
			api_key pk1_xxxxxxxxxxxx
			api_secret_key sk1_xxxxxxxxxxx
		}
		domains {
			plex.castlewolf.net @
			jelly.castlewolf.net @
			paper.castlewolf.net @
			
		}
		ip_source interface igb0
		check_interval 5m
		versions ipv4
		ttl 1h
	}

	email acme_caddy_toq@issitoq.com
	import /usr/local/etc/caddy/caddy.d/*.global
}

# Reverse Proxy Configuration


# Reverse Proxy Domain: "a44667ac-9876-48f2-89a3-ee5f88110100"
plex.castlewolf.net {
	tls /var/db/caddy/data/caddy/certificates/temp/65e77061d05bd.pem /var/db/caddy/data/caddy/certificates/temp/65e77061d05bd.key

	handle {
		reverse_proxy 192.168.123.60:32400 {
		}
	}

	abort
}
# Reverse Proxy Domain: "61f39389-3df9-42de-b6bc-9608b04cc445"
jelly.castlewolf.net {
	tls /var/db/caddy/data/caddy/certificates/temp/65e77061d05bd.pem /var/db/caddy/data/caddy/certificates/temp/65e77061d05bd.key

	handle {
		reverse_proxy 192.168.123.60:8096 {
		}
	}

	abort
}
# Reverse Proxy Domain: "3a77f88f-88ce-43ca-929e-62e699e5a5d9"
paper.castlewolf.net {
	tls /var/db/caddy/data/caddy/certificates/temp/65e77061d05bd.pem /var/db/caddy/data/caddy/certificates/temp/65e77061d05bd.key

	handle {
		reverse_proxy 192.168.123.60:8777 {
		}
	}

	abort
}

5. Links to relevant resources:

If I configure port forwards in OPENSENSE then connections from outside get routed properly but the whole point of caddy is to not have to use the port numbers correct?

2

Did you follow the requirements here, it explains the Firewall rules and setup for it to work right.

https://docs.opnsense.org/manual/how-tos/caddy.html#prepare-opnsense-for-caddy-after-installation

Also, you don’t have to use custom certificates from the acme client plugin. Caddy gets certificates automatically, it’s one of its defining features.

3

Yes, I followed those guidelines. I have rules on the WAN and the LAN interfaces directing 80 and 443 to This Firewall.

I know I don’t need to use the ACME Cert for Caddy, but I am using it for other things. It seems to work for Caddy, At least the cert is being served up on internal domains. I can’t get external ones to connect.

4

I guess DNS Problems can be ruled out, since you can resolve your external domain name from internally and connect to the domain served by caddy.

I want to know if there’s a problem with the socket, maybe overlapping ports between different services or something.

Can I see the output of:

sockstat -l -P tcp

5

Sorry, I tried running that from the command line but I don’t get any output.

admin@OPNsense:~ $ sockstat -l -P tcp
USER     COMMAND    PID   FD PROTO  LOCAL ADDRESS         FOREIGN ADDRESS      
admin@OPNsense:~ $
6

That’s weird, I would expect an output like this, you need root priviledges for the command:

I just want to make sure caddy listens on different ports than lighttpd and other services.

root@opn02:~ # sockstat -l -P tcp
USER     COMMAND    PID   FD PROTO  LOCAL ADDRESS         FOREI
GN ADDRESS
root     crowdsec   81604 13 tcp4   127.0.0.1:8080        *:*
root     crowdsec   81604 139 tcp4  127.0.0.1:6060        *:*
unbound  unbound    21091 6  tcp6   *:53                  *:*
unbound  unbound    21091 8  tcp4   *:53                  *:*
unbound  unbound    21091 37 tcp4   127.0.0.1:953         *:*
root     caddy      94122 8  tcp46  *:7861                *:*
root     caddy      94122 10 tcp46  *:80                  *:*
root     caddy      94122 11 tcp46  *:7862                *:*
root     caddy      94122 13 tcp46  *:443                 *:*
root     caddy      94122 15 tcp46  *:7860                *:*
root     lighttpd   54916 7  tcp4   *:4444                *:*
root     lighttpd   54916 8  tcp6   *:4444                *:*
root     sshd       53389 3  tcp6   *:4445                *:*
root     sshd       53389 4  tcp4   *:4445                *:*

Then afterwards, lets see if the requests actually hit the firewall. (needs also root)

ifconfig

Find out the name of your WAN interface, then enable tcpdump. Example if the interface is igc2:

tcpdump -i igc2 port 80 or port 443

Then open one of your websites and see if there is any traffic hitting the WAN interface with this request.

If that’s the case, then it’s time for the HTTP Access Log of Caddy.

Go to " Services: Caddy Web Server: General Settings" and enable “Log HTTP Access in JSON Format”. Afterwards go to one of your domains, and enable “HTTP Access Log”.

Afterwards, you can see all IP addresses that try to reach this domain in /var/log/caddy/access/UUID.log

If requests from external IP addresses appear there, then you should look into the normal Caddy Log File in the GUI, and filter for “ERROR”. Look for stuff like this, (maybe):

"error","ts":"2024-04-10T03:17:34Z","logger":"http.handlers.reverse_proxy","msg":"aborting with incomplete response","upstream":"10.0.0.20:8443","duration":0.00181995,"request":

It’s just an example of an error, there can be more errors.

If that still doesn’t show anything to work with, then I don’t really know. I guess then somebody else has to step in with in depth caddy debugging knowledge.

7

Sorry, I thought admin had root privy, it did not. Had to su to root and then run the command

USER     COMMAND    PID   FD PROTO  LOCAL ADDRESS         FOREIGN ADDRESS      
root     caddy      10534 9  tcp46  *:443                 *:*
root     caddy      10534 11 tcp46  *:80                  *:*
root     lighttpd   17298 4  tcp4   127.0.0.1:43580       *:*
unbound  unbound    64805 6  tcp4   192.168.100.1:53      *:*
unbound  unbound    64805 8  tcp4   192.168.200.1:53      *:*
unbound  unbound    64805 10 tcp4   192.168.123.1:53      *:*
unbound  unbound    64805 12 tcp6   2600:4040:7e0d:a500:227c:14ff:fea2:12f9:53 *:*
unbound  unbound    64805 14 tcp6   fe80::227c:14ff:fea2:12f9%igb1:53 *:*
unbound  unbound    64805 16 tcp4   127.0.0.1:53          *:*
unbound  unbound    64805 18 tcp6   ::1:53                *:*
unbound  unbound    64805 20 tcp6   fe80::1%lo0:53        *:*
unbound  unbound    64805 22 tcp4   192.168.100.1:53      *:*
unbound  unbound    64805 24 tcp4   192.168.200.1:53      *:*
unbound  unbound    64805 26 tcp4   192.168.123.1:53      *:*
unbound  unbound    64805 28 tcp6   2600:4040:7e0d:a500:227c:14ff:fea2:12f9:53 *:*
unbound  unbound    64805 30 tcp6   fe80::227c:14ff:fea2:12f9%igb1:53 *:*
unbound  unbound    64805 32 tcp4   127.0.0.1:53          *:*
unbound  unbound    64805 34 tcp6   ::1:53                *:*
unbound  unbound    64805 36 tcp6   fe80::1%lo0:53        *:*
unbound  unbound    64805 38 tcp4   192.168.100.1:53      *:*
unbound  unbound    64805 40 tcp4   192.168.200.1:53      *:*
unbound  unbound    64805 42 tcp4   192.168.123.1:53      *:*
unbound  unbound    64805 44 tcp6   2600:4040:7e0d:a500:227c:14ff:fea2:12f9:53 *:*
unbound  unbound    64805 46 tcp6   fe80::227c:14ff:fea2:12f9%igb1:53 *:*
unbound  unbound    64805 48 tcp4   127.0.0.1:53          *:*
unbound  unbound    64805 50 tcp6   ::1:53                *:*
unbound  unbound    64805 52 tcp6   fe80::1%lo0:53        *:*
unbound  unbound    64805 54 tcp4   192.168.100.1:53      *:*
unbound  unbound    64805 56 tcp4   192.168.200.1:53      *:*
unbound  unbound    64805 58 tcp4   192.168.123.1:53      *:*
unbound  unbound    64805 60 tcp6   2600:4040:7e0d:a500:227c:14ff:fea2:12f9:53 *:*
unbound  unbound    64805 62 tcp6   fe80::227c:14ff:fea2:12f9%igb1:53 *:*
unbound  unbound    64805 64 tcp4   127.0.0.1:53          *:*
unbound  unbound    64805 66 tcp6   ::1:53                *:*
unbound  unbound    64805 68 tcp6   fe80::1%lo0:53        *:*
unbound  unbound    64805 69 tcp4   127.0.0.1:953         *:*
root     lighttpd   32960 7  tcp4   127.0.0.1:8443        *:*
root     lighttpd   32960 8  tcp6   ::1:8443              *:*
root     lighttpd   32960 9  tcp6   fe80::1%lo0:8443      *:*
root     lighttpd   32960 10 tcp4   192.168.123.1:8443    *:*
root     lighttpd   32960 11 tcp6   2600:4040:7e0d:a500:227c:14ff:fea2:12f9:8443 *:*
root     lighttpd   32960 12 tcp6   fe80::227c:14ff:fea2:12f9%igb1:8443 *:*
root     sshd       24867 3  tcp6   fe80::1%lo0:22        *:*
root     sshd       24867 4  tcp6   ::1:22                *:*
root     sshd       24867 5  tcp4   127.0.0.1:22          *:*
root     sshd       24867 6  tcp6   fe80::227c:14ff:fea2:12f9%igb1:22 *:*
root     sshd       24867 7  tcp6   2600:4040:7e0d:a500:227c:14ff:fea2:12f9:22 *:*
root     sshd       24867 8  tcp4   192.168.123.1:22      *:*

and tcpdump

root@OPNsense:/home/admin # tcpdump -i igb0 port 80 or port 443
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on igb0, link-type EN10MB (Ethernet), capture size 262144 bytes
10:21:10.935444 IP pool-72-94-151-69.phlapa.fios.verizon.net.18843 > 151.101.46.110.https: Flags [.], ack 1243659511, win 2048, length 0
10:21:10.942022 IP 151.101.46.110.https > pool-72-94-151-69.phlapa.fios.verizon.net.18843: Flags [.], ack 1, win 424, options [nop,nop,TS val 1019892262 ecr 1897038619], length 0
10:21:11.301507 IP pool-72-94-151-69.phlapa.fios.verizon.net.18534 > 52.159.126.152.https: Flags [P.], seq 2430299025:2430299130, ack 3975323061, win 1027, length 105
10:21:11.327150 IP 52.159.126.152.https > pool-72-94-151-69.phlapa.fios.verizon.net.18534: Flags [P.], seq 1:176, ack 105, win 7247, length 175
10:21:11.507675 IP pool-72-94-151-69.phlapa.fios.verizon.net.18534 > 52.159.126.152.https: Flags [.], ack 176, win 1026, length 0
10:21:11.894597 IP6 2600:4040:7e0d:a500::1f71.55581 > lga34s34-in-x03.1e100.net.https: Flags [P.], seq 1454702368:1454702467, ack 2950211151, win 2048, options [nop,nop,TS val 2023390983 ecr 1913298263], length 99
10:21:11.903520 IP6 lga34s34-in-x03.1e100.net.https > 2600:4040:7e0d:a500::1f71.55581: Flags [.], ack 99, win 279, options [nop,nop,TS val 1913305918 ecr 2023390983], length 0
10:21:11.916342 IP6 lga34s34-in-x03.1e100.net.https > 2600:4040:7e0d:a500::1f71.55581: Flags [P.], seq 1:73, ack 99, win 279, options [nop,nop,TS val 1913305931 ecr 2023390983], length 72
10:21:11.916348 IP6 lga34s34-in-x03.1e100.net.https > 2600:4040:7e0d:a500::1f71.55581: Flags [P.], seq 73:147, ack 99, win 279, options [nop,nop,TS val 1913305931 ecr 2023390983], length 74
10:21:11.917046 IP6 lga34s34-in-x03.1e100.net.https > 2600:4040:7e0d:a500::1f71.55581: Flags [P.], seq 147:178, ack 99, win 279, options [nop,nop,TS val 1913305931 ecr 2023390983], length 31
10:21:11.917052 IP6 lga34s34-in-x03.1e100.net.https > 2600:4040:7e0d:a500::1f71.55581: Flags [P.], seq 178:217, ack 99, win 279, options [nop,nop,TS val 1913305931 ecr 2023390983], length 39
10:21:11.918914 IP6 2600:4040:7e0d:a500::1f71.55581 > lga34s34-in-x03.1e100.net.https: Flags [.], ack 217, win 2044, options [nop,nop,TS val 2023391014 ecr 1913305931], length 0
10:21:11.921281 IP6 2600:4040:7e0d:a500::1f71.55581 > lga34s34-in-x03.1e100.net.https: Flags [P.], seq 99:134, ack 217, win 2048, options [nop,nop,TS val 2023391016 ecr 1913305931], length 35
10:21:11.921287 IP6 2600:4040:7e0d:a500::1f71.55581 > lga34s34-in-x03.1e100.net.https: Flags [P.], seq 134:173, ack 217, win 2048, options [nop,nop,TS val 2023391016 ecr 1913305931], length 39
10:21:11.926336 IP6 lga34s34-in-x03.1e100.net.https > 2600:4040:7e0d:a500::1f71.55581: Flags [.], ack 173, win 279, options [nop,nop,TS val 1913305940 ecr 2023391016], length 0
10:21:12.369916 IP6 2600:4040:7e0d:a500::1fbf.62621 > usewr1-vip-bx-007.aaplimg.com.https: Flags [P.], seq 125338578:125338617, ack 2137683347, win 38, options [nop,nop,TS val 1835490916 ecr 1951133077], length 39
10:21:12.369922 IP6 2600:4040:7e0d:a500::1fbf.62621 > usewr1-vip-bx-007.aaplimg.com.https: Flags [P.], seq 39:63, ack 1, win 38, options [nop,nop,TS val 1835490916 ecr 1951133077], length 24
10:21:12.370031 IP6 2600:4040:7e0d:a500::1fbf.62621 > usewr1-vip-bx-007.aaplimg.com.https: Flags [F.], seq 63, ack 1, win 38, options [nop,nop,TS val 1835490918 ecr 1951133077], length 0
10:21:12.370036 IP6 2600:4040:7e0d:a500::1fbf.62621 > usewr1-vip-bx-007.aaplimg.com.https: Flags [FP.], seq 0:63, ack 1, win 38, options [nop,nop,TS val 1835491147 ecr 1951133077], length 63
10:21:12.370044 IP pool-72-94-151-69.phlapa.fios.verizon.net.62778 > ec2-54-163-77-158.compute-1.amazonaws.com.https: Flags [P.], seq 2202613278:2202613317, ack 3849003946, win 68, options [nop,nop,TS val 804264124 ecr 2448380695], length 39
10:21:12.370051 IP pool-72-94-151-69.phlapa.fios.verizon.net.62778 > ec2-54-163-77-158.compute-1.amazonaws.com.https: Flags [P.], seq 39:63, ack 1, win 68, options [nop,nop,TS val 804264126 ecr 2448380695], length 24
10:21:12.370059 IP pool-72-94-151-69.phlapa.fios.verizon.net.62778 > ec2-54-163-77-158.compute-1.amazonaws.com.https: Flags [F.], seq 63, ack 1, win 68, options [nop,nop,TS val 804264130 ecr 2448380695], length 0
10:21:12.370066 IP pool-72-94-151-69.phlapa.fios.verizon.net.62778 > ec2-54-163-77-158.compute-1.amazonaws.com.https: Flags [FP.], seq 0:63, ack 1, win 68, options [nop,nop,TS val 804264354 ecr 2448380695], length 63
10:21:12.371453 IP pool-72-94-151-69.phlapa.fios.verizon.net.40227 > resolver1.opendns.com.https: UDP, length 196
10:21:12.371459 IP pool-72-94-151-69.phlapa.fios.verizon.net.17628 > resolver1.opendns.com.https: UDP, length 196
10:21:12.371469 IP pool-72-94-151-69.phlapa.fios.verizon.net.1237 > ec2-54-67-119-89.us-west-1.compute.amazonaws.com.https: Flags [P.], seq 3448494010:3448494120, ack 350991542, win 2048, options [nop,nop,TS val 206188646 ecr 515387430], length 110
10:21:12.371479 IP pool-72-94-151-69.phlapa.fios.verizon.net.1237 > ec2-54-67-119-89.us-west-1.compute.amazonaws.com.https: Flags [.], seq 110:1558, ack 1, win 2048, options [nop,nop,TS val 206188646 ecr 515387430], length 1448
10:21:12.371487 IP pool-72-94-151-69.phlapa.fios.verizon.net.1237 > ec2-54-67-119-89.us-west-1.compute.amazonaws.com.https: Flags [P.], seq 1558:1762, ack 1, win 2048, options [nop,nop,TS val 206188646 ecr 515387430], length 204
10:21:12.371496 IP pool-72-94-151-69.phlapa.fios.verizon.net.1237 > ec2-54-67-119-89.us-west-1.compute.amazonaws.com.https: Flags [.], seq 1762:3210, ack 1, win 2048, options [nop,nop,TS val 206188646 ecr 515387430], length 1448
10:21:12.371508 IP pool-72-94-151-69.phlapa.fios.verizon.net.1237 > ec2-54-67-119-89.us-west-1.compute.amazonaws.com.https: Flags [P.], seq 3210:4199, ack 1, win 2048, options [nop,nop,TS val 206188646 ecr 515387430], length 989
10:21:12.371515 IP pool-72-94-151-69.phlapa.fios.verizon.net.1237 > ec2-54-67-119-89.us-west-1.compute.amazonaws.com.https: Flags [.], seq 4199:5647, ack 1, win 2048, options [nop,nop,TS val 206188936 ecr 515387430], length 1448
10:21:12.375015 IP6 usewr1-vip-bx-007.aaplimg.com.https > 2600:4040:7e0d:a500::1fbf.62621: Flags [R], seq 2137683347, win 0, length 0
10:21:12.375022 IP6 usewr1-vip-bx-007.aaplimg.com.https > 2600:4040:7e0d:a500::1fbf.62621: Flags [R], seq 2137683347, win 0, length 0
10:21:12.375027 IP6 usewr1-vip-bx-007.aaplimg.com.https > 2600:4040:7e0d:a500::1fbf.62621: Flags [R], seq 2137683347, win 0, length 0
10:21:12.376470 IP6 usewr1-vip-bx-007.aaplimg.com.https > 2600:4040:7e0d:a500::1fbf.62621: Flags [R], seq 2137683347, win 0, length 0
10:21:12.378479 IP resolver1.opendns.com.https > pool-72-94-151-69.phlapa.fios.verizon.net.17628: UDP, length 304
10:21:12.378859 IP ec2-54-163-77-158.compute-1.amazonaws.com.https > pool-72-94-151-69.phlapa.fios.verizon.net.62778: Flags [R.], seq 1, ack 39, win 68, length 0
10:21:12.378867 IP ec2-54-163-77-158.compute-1.amazonaws.com.https > pool-72-94-151-69.phlapa.fios.verizon.net.62778: Flags [R.], seq 1, ack 63, win 68, length 0
10:21:12.379235 IP ec2-54-163-77-158.compute-1.amazonaws.com.https > pool-72-94-151-69.phlapa.fios.verizon.net.62778: Flags [R.], seq 1, ack 63, win 68, length 0
10:21:12.379242 IP ec2-54-163-77-158.compute-1.amazonaws.com.https > pool-72-94-151-69.phlapa.fios.verizon.net.62778: Flags [R.], seq 1, ack 63, win 68, length 0
10:21:12.379553 IP resolver1.opendns.com.https > pool-72-94-151-69.phlapa.fios.verizon.net.40227: UDP, length 412
10:21:12.420252 IP pool-72-94-151-69.phlapa.fios.verizon.net.40046 > 93.243.107.34.bc.googleusercontent.com.https: Flags [P.], seq 348910020:348910059, ack 1828170203, win 2048, options [nop,nop,TS val 518536765 ecr 4003933940], length 39
10:21:12.420256 IP pool-72-94-151-69.phlapa.fios.verizon.net.46878 > 209.100.149.34.bc.googleusercontent.com.https: Flags [P.], seq 2476380618:2476380664, ack 902894078, win 2048, options [nop,nop,TS val 1537442676 ecr 4276236553], length 46
10:21:12.423119 IP pool-72-94-151-69.phlapa.fios.verizon.net.1237 > ec2-54-67-119-89.us-west-1.compute.amazonaws.com.https: Flags [P.], seq 5647:5975, ack 1, win 2048, options [nop,nop,TS val 206188989 ecr 515387430], length 328
10:21:12.425505 IP 209.100.149.34.bc.googleusercontent.com.https > pool-72-94-151-69.phlapa.fios.verizon.net.46878: Flags [R], seq 902894078, win 0, length 0
10:21:12.425868 IP pool-72-94-151-69.phlapa.fios.verizon.net.30847 > ec2-34-204-109-226.compute-1.amazonaws.com.https: Flags [F.], seq 2094567151, ack 48487688, win 2048, options [nop,nop,TS val 3331341400 ecr 2335747501], length 0
10:21:12.426500 IP 93.243.107.34.bc.googleusercontent.com.https > pool-72-94-151-69.phlapa.fios.verizon.net.40046: Flags [R], seq 1828170203, win 0, length 0

The only IPs I see in the log are internal

root@OPNsense:/var/log/caddy/access # cat 61f39389-3df9-42de-b6bc-9608b04cc445.log
{"level":"info","ts":1712759218.5605552,"logger":"http.log.access.log0","msg":"handled request","request":{"remote_ip":"192.168.123.166","remote_port":"62710","client_ip":"192.168.123.166","proto":"HTTP/2.0","method":"GET","host":"jelly.castlewolf.net","uri":"/System/Info/Public","headers":{"Sec-Fetch-Dest":["empty"],"Sec-Fetch-Mode":["cors"],"Sec-Fetch-Site":["same-origin"],"Te":["trailers"],"User-Agent":["Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:124.0) Gecko/20100101 Firefox/124.0"],"Accept":["*/*"],"Accept-Language":["en-US,en;q=0.5"],"Accept-Encoding":["gzip, deflate, br"]},"tls":{"resumed":false,"version":772,"cipher_suite":4865,"proto":"h2","server_name":"jelly.castlewolf.net"}},"bytes_read":0,"user_id":"","duration":0.002209471,"size":213,"status":200,"resp_headers":{"Server":["Caddy","Kestrel"],"Alt-Svc":["h3=\":443\"; ma=2592000"],"Content-Type":["application/json; charset=utf-8"],"Date":["Wed, 10 Apr 2024 14:26:58 GMT"],"X-Response-Time-Ms":["0"]}}

there are some errors in the regular log file that look like this but my IP hasn’t changed and DNS lookups for the domain point to the correct IP.

2024-04-10T10:29:36-04:00	Error	caddy	"error","ts":"2024-04-10T14:29:36Z","logger":"dynamic_dns","msg":"failed setting DNS record(s) with new IP address(es)","zone":"jelly.castlewolf.net","error":"Invalid http response status, <!DOCTYPE html>\n<html lang=\"en\">\n<head>\n<title>Error</title>\n<style type=\"text/css\">\n\n::selection{ background-color: #E13300; color: white; }\n::moz-selection{ background-color: #E13300; color: white; }\n::webkit-selection{ background-color: #E13300; color: white; }\n\nbody {\n\tbackground-color: #fff;\n\tmargin: 40px;\n\tfont: 13px/20px normal Helvetica, Arial, sans-serif;\n\tcolor: #4F5155;\n}\n\na {\n\tcolor: #003399;\n\tbackground-color: transparent;\n\tfont-weight: normal;\n}\n\nh1 {\n\tcolor: #444;\n\tbackground-color: transparent;\n\tborder-bottom: 1px solid #D0D0D0;\n\tfont-size: 19px;\n\tfont-weight: normal;\n\tmargin: 0 0 14px 0;\n\tpadding: 14px 15px 10px 15px;\n}\n\ncode {\n\tfont-family: Consolas, Monaco, Courier New, Courier, monospace;\n\tfont-size: 12px;\n\tbackground-color: #f9f9f9;\n\tborder: 1px solid #D0D0D0;\n\tcolor: #002166;\n\tdisplay: block;\n\tmargin: 14px 0 14px 0;\n\tpadding: 12px 10px 12px 10px;\n}\n\n#container {\n\tmargin: 10px;\n\tborder: 1px solid #D0D0D0;\n\t-webkit-box-shadow: 0 0 8px #D0D0D0;\n}\n\np {\n\tmargin: 12px 15px 12px 15px;\n}\n\n</style>\n</head>\n\n\n<body>\n\t<div id=\"container\">\n\t\t<h1>An Error Was Encountered</h1>\n\t\t<p>The URI you submitted has disallowed characters.</p>\t\t<p>You can probably find what you're looking for on our <a href=\"/\">homepage</a>.</p>\n\t</div>\n</body>\n</html>\n"}

I’m guessing it’s something stupid or a conflict. But any additional ideas of what to look for would be appreciated. Thanks,

8

If https traffic hits the WAN interface when you open one of your domains, but doesn’t appear in the HTTP access log, then it’s most likely a firewall or NAT issue.

Double check the Firewall rules on your WAN interface. Look at the

Firewall: Log Files: Live View

To see if incoming packets are dropped that have 80 or 443 as port and your external IP address as destination.

Then also check for old Port Forward rules. Disable them all for troubleshooting.

For further troubleshooting, you could create a rule in “Firewall: Rules: Floating” which allows anything to the external IP address of your firewall. Put it at the first spot in that list and don’t choose any interface explicitely, just leave that empty, as Destination you input the IPv4 address of your Firewall (external IP) without any alias. Don’t forget to disable it again.

EDIT: Maybe with dynamic IPs the “This Firewall” alias doesn’t work right, try to use the “interface_name address” alias instead. (mine is called igc1 address, or WAN address)

9

Wow, Live View is something else. New to OPNSense, never seen that before. Thanks.

I can see my external requests hitting the firewall and then getting denied with this error “Default deny / state violation rule” Trying to figure that out now. This may be an OPNSense question as opposed to Caddy.

10

Yeah best come to the OPNsense forum. Link this thread there and ask for help.

It definitely sounds like a firewall or configuration issue of the OPNsense.

1 Like
11

Thanks for helping @Monviech, I don’t have any experience with OPNSense.

I agree it sounds like a networking config issue, not a problem with Caddy specifically.

1 Like
12

This one is actually pretty interesting. I think there is a client side redirect going on.

Thats why it looked like theres no connection on WAN, since only port 443 was allowed, so the browser hang because it couldnt follow the redirect to port 5001.

I don’t know how to help there.

13

O. M. G.

I knew it was something stupid simple that I was missing. Thank you for your comment. I had port forwards in place for 80 and 443. (Firewall → NAT → Port Forward) Removed those, everything works as expected.

My Synology was handling reverse proxy previously which is why I forwarded the ports. Then I switched to nginx but never got it working properly (because of the port forwards) but it was so complicated I never realize what was going on or even what/who/how to ask for help. When I switched to caddy and got the internal working flawlessly I figure it was something I was missing. And I was thanks. Thanks for the mental ping of where to look.

1 Like
14

That’s awesome. Happy you got it to work finally.

Read a bit further up, I suggested that too.

“Then also check for old Port Forward rules. Disable them all for troubleshooting.”

:blush:

15

I had mistaken my firewall rules for Port Forwards and mistakenly assumed they were disabled.

Thanks again.

1 Like
16

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.