Rewrites not working?

Help
1

1. The problem I’m having:

I’m trying to make a reverse proxy to a certain website on a specific path, in other words, I want “mydomain.com” to show the contents of “42 (number) - Wikipedia”. Instead of this, the server seems to show only the contents of the website I’m proxying, no path.

2. Error messages and/or full log output:

There are no error messages

3. Caddy version:

v2.8.4 with docker compose

4. How I installed and ran Caddy:

Docker on archlinux

a. System environment:

Arch Linux x86_64, 6.10.8-arch1-1, Docker version 27.2.0

b. Command:

docker compose up -d

c. Service/unit/compose file:

  caddy:
    image: caddy:latest
    container_name: caddy
    restart: unless-stopped
    ports:
      - "80:80"
      - "443:443"
      - "443:443/udp"
    volumes:
      - ~/docker/caddy/Caddyfile:/etc/caddy/Caddyfile
      - ~/docker/caddy/caddy_data:/data
      - ~/docker/mystery/:/var/www/mystery

d. My complete Caddy config:

mydomain.com {
 rewrite * /wiki/42_(number)
 reverse_proxy https://en.wikipedia.org
}
2

Howdy @dmitridut, welcome to the Caddy community.

I did a quick test and was able to get a similar rewrite working without issue. (Well, the page had a few issues, but the rewrite itself was fully functional.)

Please add the debug global option to your Caddyfile, make a few requests, and post the resulting upstream logs here.

1 Like
3

Keep in mind that a single webpage may be multiple actual requests (including CSS, JS, etc) so if you rewrite every request to a specific URL, then it would likely break assets trying to be loaded by that page as well.

1 Like
4 
{"level":"debug","ts":1726535773.6507215,"logger":"http.handlers.reverse_proxy","msg":"upstream roundtrip","upstream":"en.wikipedia.org:443","duration":0.142004199,"request":{"remote_ip":"186.11.13.128","remote_port":"57861","client_ip":"186.11.13.128","proto":"HTTP/2.0","method":"GET","host":"mydomain.com","uri":"/wiki/42_%28number%29","headers":{"User-Agent":["DuckDuckGo/5 (com.duckduckgo.mobile.android; Android API 33)"],"Accept-Encoding":["gzip"],"X-Forwarded-For":["186.11.13.128"],"X-Forwarded-Proto":["https"],"X-Forwarded-Host":["mydomain.com"]},"tls":{"resumed":false,"version":772,"cipher_suite":4865,"proto":"h2","server_name":"mydomain.com"}},"headers":{"X-Cache":["cp1106 int"],"X-Cache-Status":["int-front"],"Set-Cookie":["REDACTED"],"Date":["Tue, 17 Sep 2024 01:16:13 GMT"],"Server":["Varnish"],"Server-Timing":["cache;desc=\"int-front\", host;desc=\"cp1106\""],"X-Client-Ip":["181.43.24.97"],"Content-Type":["text/html; charset=utf-8"],"Content-Length":["1896"]},"status":400}                        {"level":"debug","ts":1726535773.7032876,"logger":"http.handlers.rewrite","msg":"rewrote request","request":{"remote_ip":"186.11.13.128","remote_port":"57861","client_ip":"186.11.13.128","proto":"HTTP/2.0","method":"GET","host":"mydomain.com","uri":"/favicon.ico","headers":{"User-Agent":["DuckDuckGo/5 (com.duckduckgo.mobile.android; Android API 33)"],"Accept-Encoding":["gzip"]},"tls":{"resumed":false,"version":772,"cipher_suite":4865,"proto":"h2","server_name":"mydomain.com"}},"method":"GET","uri":"/wiki/42_%28number%29"}         {"level":"debug","ts":1726535773.7034838,"logger":"http.handlers.reverse_proxy","msg":"selected upstream","dial":"en.wikipedia.org:443","total_upstreams":1}                                    {"level":"debug","ts":1726535773.84597,"logger":"http.handlers.reverse_proxy","msg":"upstream roundtrip","upstream":"en.wikipedia.org:443","duration":0.142250225,"request":{"remote_ip":"186.11.13.128","remote_port":"57861","client_ip":"186.11.13.128","proto":"HTTP/2.0","method":"GET","host":"mydomain.com","uri":"/wiki/42_%28number%29","headers":{"Accept-Encoding":["gzip"],"X-Forwarded-For":["186.11.13.128"],"X-Forwarded-Proto":["https"],"X-Forwarded-Host":["mydomain.com"],"User-Agent":["DuckDuckGo/5 (com.duckduckgo.mobile.android; Android API 33)"]},"tls":{"resumed":false,"version":772,"cipher_suite":4865,"proto":"h2","server_name":"mydomain.com"}},"headers":{"X-Client-Ip":["181.43.24.97"],"Content-Type":["text/html; charset=utf-8"],"Content-Length":["1896"],"Date":["Tue, 17 Sep 2024 01:16:13 GMT"],"X-Cache-Status":["int-front"],"Server-Timing":["cache;desc=\"int-front\", host;desc=\"cp1106\""],"Server":["Varnish"],"X-Cache":["cp1106 int"],"Set-Cookie":["REDACTED"]},"status":400}

is this good enough? it seemed like it made a lot of requests

could you try proxying the same page I’m trying to proxy?

so should I only rewrite to “path /” ? it doesn’t seem to be working either. maybe what I’m trying to accomplish isn’t that simple?

5

I did. The rewrite worked for me. It’s also working for you, from your logs:

{
  "level": "debug",
  "ts": 1726535773.7032876,
  "logger": "http.handlers.rewrite",
  "msg": "rewrote request",
  "request": {
    "remote_ip": "186.11.13.128",
    "remote_port": "57861",
    "client_ip": "186.11.13.128",
    "proto": "HTTP/2.0",
    "method": "GET",
    "host": "mydomain.com",
    "uri": "/favicon.ico",
    "headers": {
      "User-Agent": [
        "DuckDuckGo/5 (com.duckduckgo.mobile.android; Android API 33)"
      ],
      "Accept-Encoding": [
        "gzip"
      ]
    },
    "tls": {
      "resumed": false,
      "version": 772,
      "cipher_suite": 4865,
      "proto": "h2",
      "server_name": "mydomain.com"
    }
  },
  "method": "GET",
  "uri": "/wiki/42_%28number%29"
}

Your browser started out requesting /favicon.ico and Caddy corrected it to /wiki/42_%28number%29 (which is just /wiki/42_(number) but URL-encoded).

An issue arises when the website relies on additional assets (like the favicon shown above, as well as stylesheets, images, javascript etc) that the HTML document links to. You’ve configured Caddy to rewrite everything, so all of these also get redirected to requests for the main page document instead of the extra assets.

This will make the page “look” broken, but it should still work. You wrote in your original post that it shows only the contents of the website with no path - for me, it shows it with the correct path, but naturally without any assets.

It’s less simple than a straightforward reverse proxy, for sure.

You might consider only rewriting from /, or maybe consider a redirect instead.

1 Like
6

Here’s how it looked for me, FYI.

42
422126×1436 339 KB

My Caddyfile was:

localhost {
  rewrite * /wiki/42_(number)
  reverse_proxy https://en.wikipedia.org {
    # Wikipedia servers didn't like Host localhost
    header_up Host en.wikipedia.org
  }
}
1 Like
7

This topic was automatically closed 30 days after the last reply. New replies are no longer allowed.