I want to add a custom caddy.App to do some configuration (db, redis and some others), then I will use them in other Apps like http, for example a rate limit plugin http.handlers.ratelimit, which will use redis, but it need to ensure the config App load before http App, but now it seems that the AppsRaw is ModuleMap, It doesn’t guarantee order, so I can not validate in Validate method of ratelimit Module, Is there any other way to specify apps load order?
thanks
Great question!
Eventually I want to write up some more about integrating with Caddy, but have been too focused on its development in the meantime.
A Caddy app is something that you’d typically Start() and then Stop(). It’s a specific kind of Caddy module with that interface. I don’t really know what you mean by “do some configuration”. Would it be a good fit for that interface?
You can probably do this without a redis Caddy app. I think a Caddy app would make sense if it was the database itself, but if it’s just a connection to a database, an entire app module may not be necessary. Why not just have your HTTP handler module open a redis connection?
(PS. I have plans to write an official rate limit handler at some point, in case that’s of interest, maybe we should collaborate on a single design.)
Correct, apps are loaded in an arbitrary order (if they are even loaded at all). I am actually looking at this situation either this or next week as I am writing a new Caddy app as well.
However, in your case, I don’t think you need an App. It sounds like you just need a database connection, which can be done without an App module.
Yes, I agree with you, maybe App is not the suitable for this, and actually the Start and Stop is empty for my config App, I just use this as a extension point to define some resources, initialize them and then save in the registry, then in some other modules, I just want to reference it by a name, not define all needed information multiple places. for example,
{
"apps": {
"config": {
"database": {
"go-redis": {
"dbs": [
{
"id": "default",
"client_options": {
"db": 0,
"addr": "127.0.0.1:6379",
"password": "",
"max_retries": 1,
"dial_timeout": "5s",
"read_timeout": "5s",
"pool_size": 10,
// ...
},
"codec_options": {
"use_redis_cache": true,
"use_local_cache": true,
"local_cache_max_items": 5000,
"local_cache_expiration": "30s"
}
}
]
}
}
}
}
}
then for example, in a ratelimit Module, I just want reference it by name “default”, like this,
{
// ...
"routes": [
"match": [
// ...
],
"handle": [
{
"handler": "ratelimit",
"limiter_name": "default",
"duration": "1m",
"limit": "20000",
"request_extractor": {
"use_path": true,
"use_arguments": [
"channel"
]
}
}
// ...
]
]
// ...
}
Here I don’t want to define the connection parameters repeatedly，but just reference it by this "limiter_name: "default", because I might define different ratelimit in multiple places, it will make the config too verbose.
I didn’t find any other way to solve it, if there is, pls let me know.
Yeah, I’ve been running into a similar issue with a project on my end.
Right now, modules all Loaded, Provisioned, and Validated (LPV’ed) all at once: when you call LoadModule(), it loads it, provisions it, and validates it, before going on to the next app.
I was thinking of adding another method / interface that modules could implement that would be invoked after all apps have been LPV’ed, but before they’re started. Maybe called Ready(). This would allow you to get a handle on their provisioned config. Of course, the order of these invocations would also be arbitrary (unordered), so you’d have to consider that: their configs could continue to change if other modules access and change the same app during their post-LPV/pre-Start phase (tentatively called Ready).
I’m still trying to decide if that would work for me. Will that work for you?
Yes, it can solve my problem. But I think if it’s necessary to add another method like Ready, actually, I thought the Validate will be invoked after all apps have been LP’ed, but found LPV’ed all at once, I don’t understand the extra benefit of the separation(Provision & Validate), why not a Initialize(=Provision+Validate)，could you please help to explain the design intent. And if this is feasible we don’t add Ready, let Validate be invoked after apps have been LP’ed?
That’s a good point.
I think it was because in the early days of designing Caddy 2, the idea of validating a config was different from provisioning it. We have caddy validate and caddy adapt --validate commands which provision and validate a config, as opposed to just creating a config.
But you’re right, there’s no reason why a module can’t do its validation inside of Provision(). I might be willing to remove the extra Validate step and just encourage modules to implement it inside their Provision phase, since they happen at the same time.
Well, I want LoadModule() to return an error if Validate() fails, so it needs to happen near the same time as Provision().
I’m currently experimenting with a Ready() phase locally… there is one problem, which is this scenario:
Ready() to access App A (the order is arbitrary, however)I do not know if this will be a problem in practice, and maybe the best practice is that config references are exactly that: references (pointers), so you can always have the current config, but… if you have to make a decision at Ready-time, the config you’re looking at then might not yet be complete if other apps mutate it.
It’s a tricky problem… but if we are OK with some documented rule/convention that “other apps’ configs may change during Ready if other modules mutate them during Ready, before or after your module’s Ready function is called”, then this might still be fine. But it’s kind of a big caveat.
As for my config App, the resources provisioned also register in a registry, then in other Apps, I will use reference them by name, but I need ensure the reference name has been configured correct, so I need in Ready to validate if the reference name default (e.g. limiter_name: default) is valid, if invalid, the load should fail, not after App Start for example, in ServeHTTP， panic because invalid nil pointer or get some HTTP 500 like this.
I think the essential need is the apps order, once we decide that should support Apps relations, even we have documented rule/convection, users also have no choices. Whether to consider adding a field e.g. AppsReadyOrder in Config? If specified then we can execute Ready by specified order?
Okay, yeah, I have the same problem right now! 
I’ve been mulling over various solutions all this week because it’s kind of tricky.
I think I’ve found a way to make it work, implicitly, without any other changes.
When you call ctx.App() to get the reference to an app module, that method will simply return the reference to the loaded app module if it was already provisioned. If it wasn’t provisioned yet, it will take the raw JSON input and provision it, then return it. You can call ctx.App() in your Provision() method like you’d expect.
As long as there are not cyclic app dependencies (similar to how Go does not allow circular imports), this should work fine I think!
I just tested it here and it seems to be working fine. I’ll commit as soon as I’m able.
I think this will solve your problem (which is the same as my problem).
Great solution! I like it!!!
Thank you for your kindly replies.
This topic was automatically closed 90 days after the last reply. New replies are no longer allowed.