Reverse proxy for Jupyter Notebook

[nickdaves (Nick)]

1. Caddy version (caddy version):

v2 (caddy:latest tag from Docker Hub)

2. How I run Caddy:

Caddy runs in Docker container described in the next section.

a. System environment:

A VPS from Oracle, rocking Ubuntu with aarch64 CPU. Runs Docker and I prefer to manage containers via Portainer.
Caddy is set up via a stack created on Portainer.
Another way to run the stack if you don’t have Portainer is
docker-compose up

b. Command:

I just hit the button “Run stack” in Portainer.

c. Service/unit/compose file:

The YAML file contains the below script. I simplified the script, removing environment variables for the Jupyter container.

version: "3.7"
networks:
  web: # name of network for exposing caddy to the world
    external: true
  internal: # name of network for containers to be behind caddy
    external: false
    driver: bridge # all services are inter-connected (I guess?)
services:
  caddy:
    image: caddy:latest
    container_name: admcaddy
    restart: unless-stopped
    ports:
      - 80:80
      - 443:443
    volumes:
      - /data/caddy/Caddyfile:/etc/caddy/Caddyfile # mounts Caddyfile to configure before startup
      - /data/caddy/data:/data
      - /data/caddy/config:/config
    networks:
      - web
      - internal
  datascience_jup: # the container I'm interested to expose, this name is important for caddy to work
    image: jupyter/scipy-notebook
    container_name: datascience_jup
    volumes:
      - /home/ubuntu/jupyter:/home/jovyan # folder in host mounted to container folder
    ports:
      - 8888:8888 # actually Caddy works with the internal container port (the later)
    networks:
      - internal # make jupyter container exists only in "internal" network

d. My complete Caddyfile or JSON config:

{
    # Global options block. Entirely optional, https is on by default
    # Optional email key for lets encrypt
    email myuser@sirdeniel.com
    # Optional staging lets encrypt for testing. Comment out for production.
    # acme_ca https://acme-staging-v02.api.letsencrypt.org/directory
}
sirdeniel.com {
    reverse_proxy /jupyter datascience_jup:8888
    # here I may need some parameters to make it work under /jupyter
}

3. The problem I’m having:

Even though I use c.NotebookApp.base_url = '/jupyter/' to configure Jupyter Notebook, I cannot access to it. By itself, it redirects to /jupyter/lab? but the logs from Jupyter Notebook says 404 GET /api

4. Error messages and/or full log output:

Jupyter Notebook logs says 404 GET /api. There seems to be no problem from Caddy. But the responses are always empty body aka. blank page. I suspect Jupyter Notebook itself tries to access localhost:8888/api but that points to nowhere know by Caddy, so the it defaults to blank page.

5. What I already tried:

Tweaking the reverse proxy.

    # this config seems the same to me as the original one-liner
    handle_path /jupyter/* {
        rewrite * /jupyter/{uri}
        reverse_proxy datascience_jup:8888
    }

Overall I’m a newbie user, I do not know many concepts for using proxy but would be glad to read some information about that, even better if Caddy oriented.

My only intention is to serve my Jupyter notebook under sirdeniel.com/jupyter/{uri} where {uri} corresponds to any redirection made by Jupyter notebook.

6. Links to relevant resources:

Caddy documentation
Gist for Jupyter Notbook working with Caddy prior to v2

[francislavoie]

Path matching is exactly in Caddy, so /jupyter will only match exactly /jupyter and nothing else. If you use /jupyter*, it would match other paths as well, like /jupyter/lab.

Do you need to run it under a subpath though? If you have nothing else to serve on that domain, you could just serve it from the root of the domain. Or you could use a subdomain like jupyter.sirdeniel.com.

[nickdaves (Nick)]

Thanks! It worked so well. Here is the code block I used in case someone wants to know.

sirdeniel.com {
    reverse_proxy /jupyter* datascience_jup:8888
}

Yes, I plan to have more services. I’m currently running Filebrowser but not under Caddy (I’d like to) and in the near future a blog with Ghost or static page with Hugo.
Another question is, how do I serve a subdomain like jupyter.sirdeniel.com and docs.sirdeniel.com within the same IP. Does Caddy handle that if I set it up from my registrar?

[francislavoie]

Yeah, you just point that subdomain to Caddy, then configure Caddy with that domain. For example:

jupyter.sirdeniel.com {
    reverse_proxy datascience_jup:8888
}

docs.sirdeniel.com {
    reverse_proxy datascience_docs:8000
}

[nickdaves (Nick)]

Worked like a breeze, thank you so much for the quick and assertive responses, I love Caddy!