Import: Space in File-paths

Help
#1

Well, this is about how to handle (space) in file-paths.
[I’m talking about `import` but it’s the same with everything else having a path; like `root` or the `log` directives.]

1. Caddy version (caddy version):

v2.1.1 h1:X9k1+ehZPYYrSqBvf/ocUgdLSRIuiNiMo7CvyGUQKeA=

2. How I run Caddy:

a. System environment:

Linux x64

b. Command:

caddy run

c. Service/unit/compose file:

none

paste full file contents here

d. My complete Caddyfile or JSON config:

# any Caddyfile content

import ~/myPath/with some space/in-it.conf

# any Caddyfile content

3. The problem I’m having:

Well, Caddy doesn’t behave well with spaces in paths because space is special.

4. Error messages and/or full log output:

Error during parsing: File to import not found: ~/myPath/with some space/in-it.conf

5. What I already tried:

I already tried using %20 or %A0 instead of space within the path. But it didn’t helped.
And many other things I already forgot.

6. Links to relevant resources:

The docs for import, what else :wink:

Maybe this too:

#2

You can wrap it in quotes:

#3

I tried that too … and it didn’t worked/helped :confused:

. . .

Ehhh, doesn’t matter anymore. I already wrote a script which replaced every with _ for me.
Caddy already took way too much time to setup.

#4

btw. is it a bug that I can’t rely on snippets? They aren’t well documented and sometimes they work, sometimes not.
It was way easier to replace every snippet by a static file in the same folder than trying to get snippets to work. That took me hours too. (I thought Caddyfiles could look pretty. But obviously, they won’t if you try to be 100% modular with your config.)

In the end, I found a post weeks later, that they only work in the main Caddyfile and not in any imported file. Although they did … sometimes.

#5

You didn’t show any evidence that it might be. What do you mean?

Works perfectly fine for me:

$ tree
.
├── Caddyfile
└── dir with spaces
    └── respond.conf

1 directory, 2 files

Caddyfile:

:80

import "dir with spaces/respond.conf"

respond.conf:

respond "Hello World!"

$ caddy adapt --config Caddyfile --pretty
{
	"apps": {
		"http": {
			"servers": {
				"srv0": {
					"listen": [
						":80"
					],
					"routes": [
						{
							"handle": [
								{
									"body": "Hello World!",
									"handler": "static_response"
								}
							]
						}
					]
				}
			}
		}
	}
}
#6

It’s always the same thing. When testing, it works, and on the actual system it doesn’t :joy:

Yes, the quotes worked for me too. But then it didn’t and it’s all just weird.
Can’t reproduce the quote thing now as my script already “fixed” everything :slight_smile:

I had a Caddyfile which was imported by my main Caddyfile:

(snippet) {
# I forgot what I put here, but doesn't matter I guess
}

(snippet2) {
# I forgot what I put here, but doesn't matter I guess
}

localhost:8001 {
 reverse_proxy * https://example.com {
        import snippet
 }
}

localhost:8002 {
 reverse_proxy * https://example2.com {
        import snippet2
 }
}

The weird thing is. One worked, the other one didn’t.
(With the error “couldn’t find file snippet2”.)

Again: After days of issuing the problem I just used more files instead of snippets. They are more reliable than snippets. (At least as long as they are in the same directory =)

#7

Works for me…

Please give me a clear and exact example where you’re seeing this behaviour (with file structure and all), because at this point, I’m not seeing an issue.

(snippet) {
    header_up Foo bar
}

(snippet2) {
    header_up Bar baz
}

localhost:8001 {
    reverse_proxy * https://example.com {
        import snippet
    }
}

localhost:8002 {
    reverse_proxy * https://example2.com {
        import snippet2
    }
}

$ caddy adapt --config Caddyfile --pretty
{
	"apps": {
		"http": {
			"servers": {
				"srv0": {
					"listen": [
						":8001"
					],
					"routes": [
						{
							"match": [
								{
									"host": [
										"localhost"
									]
								}
							],
							"handle": [
								{
									"handler": "subroute",
									"routes": [
										{
											"handle": [
												{
													"handler": "reverse_proxy",
													"headers": {
														"request": {
															"set": {
																"Foo": [
																	"bar"
																]
															}
														}
													},
													"transport": {
														"protocol": "http",
														"tls": {}
													},
													"upstreams": [
														{
															"dial": "example.com:443"
														}
													]
												}
											]
										}
									]
								}
							],
							"terminal": true
						}
					]
				},
				"srv1": {
					"listen": [
						":8002"
					],
					"routes": [
						{
							"match": [
								{
									"host": [
										"localhost"
									]
								}
							],
							"handle": [
								{
									"handler": "subroute",
									"routes": [
										{
											"handle": [
												{
													"handler": "reverse_proxy",
													"headers": {
														"request": {
															"set": {
																"Bar": [
																	"baz"
																]
															}
														}
													},
													"transport": {
														"protocol": "http",
														"tls": {}
													},
													"upstreams": [
														{
															"dial": "example2.com:443"
														}
													]
												}
											]
										}
									]
								}
							],
							"terminal": true
						}
					]
				}
			}
		}
	}
}
#8

Yes, I know!! It works when writing it down as a single thing.
When creating the test & examples it always worked too!
That was the weird thing.
I checked the names, the spacing, the order (having snippets at the top) etc. etc. but there were these problems.

As far as I could figure out this was the main problem:

In the main Caddyfile, it always worked. But in the import of the import, it didn’t.

(And I’m sure I had no snippet named identically in other Caddyfiles imported by the main Caddyfile. If that could be an issue.)

#9

Please give me an exact example where you can reproduce this behaviour. Every file and their contents, like I did above.

#10

You want me to reproduce smth. I fixed days ago? … (fixed = using files instead of snippets)
Sry. but as I said. In examples (reduced complexity…) it always worked, therefore, it won’t be easy to reproduce.
Additional the exact example would mean over 20 different Caddyfiles files :sweat_smile: (without the extra files for the snippets, cough)

#11

Ah, but I would have an exact example of smth. else related to imports (which I already fixed too…).
[As I don’t want to include my 20++ files here I’m trying to keep it short.]

Files:

.
├── Caddyfile
└── dir with spaces
    └── import-suff.conf
└── modules
     └── tls.conf

.
└── domainFolder
|     └── Caddyfile.conf
└──domainFolder2
     └── Caddyfile.conf


{
debug
}

import "dir with space/import-stuff.conf"
import "full-path/to/domain/folder/domainFolder/*.conf"
import "full-path/to/domain/folder/domainFolder2/*.conf"

# domainFolder/Caddyfile

localhost:80001 {
import ./modules/tls.conf # <--- Doesn't work.
}

My fix: Use of environment variables for an absolute path back to the modules folder :slight_smile:

#12

Yes because you’re claiming there’s a bug, and if there is, I want to know what it is so I can fix it.

I’m asking you to reduce the problem to as simple of a reproducible example as you can, to show how and why it happens.

#13

If domainFolder/Caddyfile is deeply nested, then that path needs to be relative to THAT file, not to the root file. But I’m not sure I understand your example because it’s a bit messy.

No pseudocode please, I want something concrete, that I can test for myself.

#14

That is my problem. Most of the time everything is related to the core-Caddyfile. (like the root directive).
(Because imports are placed one by one into the original Caddyfile.
From the Docs: “Includes a snippet or file, replacing this directive with the contents of the snippet or file.”)

But then, out of nowhere, it’s based on your current directory. It’s not consistent.

I know there were attempts (PRs and, I don’t find the -newer- issue again right now.) to have paths from your current file. So you can import them more easily and modular. But: That doesn’t apply for everything and it’s unclear when it applies.

#15

It’s always based on the current directory of the Caddyfile the import statement appears in. It is consistent.

That’s from Caddy v1. All documentation relating to Caddy v1 is outdated and irrelevant, Caddy v2 is a complete rewrite from the ground up.

That said, that PR was a fix to ensure that relative paths are resolved before applying the glob. This only reinforces my point above that import from files will always be relative to the file it appears in, not relative to the root Caddyfile.

#16

To split it up…

^ Relative to the main-Caddyfile. And nearly everything else too.

^ Relative to the current Caddyfile

Please show me the docs explaining both behaviours. I was only able to find one of them and assumed they behave the same until I noticed they don’t.

And if they behave the same, then I can’t explain the behaviour of my Caddy at all as it then has no coincidence with the docs.

#17

root is evaluated at runtime, not at config parsing time. They aren’t the same concepts.

To clarify, the Caddyfile in v2 is a config adapter, meaning it’s a translation layer to JSON config.

import is unique to the Caddyfile adapter and is meant as a code reuse aid. The root directive has nothing to do with configuration parsing. It’s a handler that sets the root variable that gets used by various handlers like file_server to tell those handlers where to find files.

I recommend using absolute paths for root to avoid the confusion.

#18

Yes, I know. That doesn’t mean the paths can’t be replaced when including them.
I assumed they are replaced correctly, but they aren’t.

edit.: Can’t create more replies, bc I count as ‘new user’ :joy:

I already do. (?)
So do I with many other things. Thankfully environment variables work…

edit2.: (Edits still work)
I would like to finish this here. Not just takes it way too much time now and I can’t create more replies… (and we nearly talked only about already fixed problems - some with indirection, but it works…)

Thanks you for your replies (I guess)…
Wasn’t pretty helpful for me now as I already took hours into these to issue and solve them… but I appreciate your replies. At the end the replies here are much faster and helpful than on GitHub… I guess…

My final words (very unrelated to this topic/the posts above. But I want to place this somewhere…):
Caddy isn’t as fast to set-up as it says. (as soon as you want more than what an example gives you - and there are only a few examples in the docs.)
As I already heard from other users CaddyV1 was easier to set up, but less powerful.
I upgraded from a CaddyV1 Setup which I didn’t write and v1 was more intuitive.
Additional I wanted to use what v2 can give me. But it’s pretty clear to me. CaddyV2 isn’t very polite/finished. Not just the docs which are written as short and minified as possible. V1 seemed to be the same (even less clear) but v1 had the userbase with forum posts and issues on GitHub which solved pretty much every problem. It wasn’t hard to find a solution for a problem for v1 or even v0 but it was for v2. (And yes, everything is easy&fast to setup if you know it very well.)

In the end: Wait 2-6 months before using caddy v2 - then there should be enough posts with solutions (and bug-fixes…). I’m still waiting for 2.2.

BB. and have fun^.

#19

That wasn’t a correct assumption to make, ultimately.

I just opened a PR to clarify the import docs to clarify that import statements are relative to the file they appear in.

https://github.com/caddyserver/website/pull/78

I find this kinda rude. I’m spending time out of my day to help, for free.

I’m glad that we were able to find the misunderstanding though.

I mean, that’s totally dependent on your usecase. It seems you’re trying to do something particularly advanced if you need as many as 20 separate config files. The ease of use will obviously scale with the complexity of your system. That’s not something Caddy (or any other webserver) can magically solve.

That may be true. Caddy v1 made all kinds of assumptions that made it inflexible in many ways. Certain things were completely impossible in Caddy v1. None of the issues with import you encountered here would have been different in Caddy v1 though, import worked largely the same way in Caddy v1.

Caddy v2 was a rewrite from the ground up to avoid those assumptions and implement a design based on the learnings of the years prior that would be more flexible and usable for as many usecases as possible.

That’s totally dependent on the usecase, for sure. Some concessions had to be made in v2 to make more things possible. What specifically did you find less intuitive, out of curiosity?

Caddy v2 is much more polished than v1. There’s absolutely no question there. It can do so much more.

I disagree, the docs are very verbose and cover everything you would need to know. I concede the behaviour of import was not as clear as it should have been, but that’s a very minor issue all things considered.

That’s just an issue of time. Since Caddy v2 was only released 6 months ago, there is definitely less content than the ~4-5 years of content accumulated for Caddy v1. But really, you should look at the date of any issue or article you find to see whether it’s relevant or not. Nothing we can do to help you there.

1 Like
#20

This topic was automatically closed after 30 days. New replies are no longer allowed.