Using internal caddy structures in other golang apps

flowbart · March 4, 2024, 8:38pm

Hi everybody,

currently i try to develop a app which is unsing the caddy api to establish configuration changes dynamically during runtime from another golang application.

The application should be able to change certain things on a running caddy instance using its api. The app is offering some abstracted rest api with certain other things which are not related to the reverseproxy on a single server.

Maybe also because im quite new to golang i’m having some difficulties regarding the mapping of the dynamic caddy datastructures during runtime.

The idea is to hide certain fields for the user of the original caddy API and offer only a few features and not all fieldsexposed because they will be preconfigured and just internal.

I thougth about just reusing the caddy structures (e.g. caddytls.acmeissuer) in the client code of the golang application and unmashall the data directy in the respective structures. But its not that easy because of a lot of polymorhic which makes the parsing verry complex.

I would like to avoid to “reinvent the wheel” and looking for something which can be reused. I was thinking caddy has to deal with simmilar problems even if it is in reverse order.

As example i would take a look at a polymorphic stucture like JSON Config Structure - Caddy Documentation

A “internal issuer” is quite easy to unmarshal from the received json request but when there is for example “acme issuer” its getting komplex.

What could be the best way to archiving this kind of abstraction of the caddy API?
I want to be able to do all necessary request ( GET / POST / PUT…) to caddy via the standard api.

Example:

“GET”, “apps/tls/automation” to caddy.socket
unmarshal request into caddy sturctures
map relevant data into structures represented by the client application api (api schemas of webserver endpoints)

I need some inspiration and feedback as well. Maybe the idea is dump?! How would you deal with that? What would be youre recommendation?

Thanks in advance!

matt · March 5, 2024, 4:45am

You can reuse the data types that Caddy does if you’re writing Go. The “polymorphic” fields, which are module fields, get unmarshaled into json.RawMessage. You can then proceed to unmarshal that depending on the type/value that fills that space. The tricky part is knowing what type to unmarshal each module config into.

The way Caddy unmarshals module JSONs dynamically is in context.go in the LoadModule()-family-of-functions:

github.com

caddyserver/caddy/blob/8f8204708ac5f1dfdd0ffa7bcad1974ec54e18e9/context.go#L157


      
          // in order to know the module name.
          //
          // To make use of the loaded module(s) (the return value), you will probably want
          // to type-assert each 'any' value(s) to the types that are useful to you
          // and store them on the same struct. Storing them on the same struct makes for
          // easy garbage collection when your host module is no longer needed.
          //
          // Loaded modules have already been provisioned and validated. Upon returning
          // successfully, this method clears the json.RawMessage(s) in the field since
          // the raw JSON is no longer needed, and this allows the GC to free up memory.
          func (ctx Context) LoadModule(structPointer any, fieldName string) (any, error) {
          	val := reflect.ValueOf(structPointer).Elem().FieldByName(fieldName)
          	typ := val.Type()
          
          	field, ok := reflect.TypeOf(structPointer).Elem().FieldByName(fieldName)
          	if !ok {
          		panic(fmt.Sprintf("field %s does not exist in %#v", fieldName, structPointer))
          	}
          
          	opts, err := ParseStructTag(field.Tag.Get("caddy"))
          	if err != nil {

I admit it is not pretty. But it works quite well. Unfortunately it doesn’t generalize well outside of Caddy. Maybe it could be refactored so that an exported API makes it easier to use outside of Caddy.

Does that help?

flowbart · March 5, 2024, 6:01am

Hi Matt,

Thank you for your prompt reply and for your commitment to this excellent project. Your response was indeed helpful.

I was in search of a specific API, but it appears it’s not available at this time. It would be fantastic if the API could be released in the future. Additionally, optimizing the caddy implementation of the unmarshal operations with this approach could streamline the process, eliminating the need for any admissions though, in all fairness, there’s nothing to admit in the first place .

I’ll will take a look at the context.go and the LoadModule-functions.

Thank you!

francislavoie · March 5, 2024, 6:35am

Have you seen GitHub - abiosoft/caddy-json-schema: JSON schema generator for Caddy v2 ? You might be able to generate a JSON schema then turn that into Go structures with some other tool.

flowbart · March 5, 2024, 7:23am

Hi @francislavoie,

im not sure if i understand your suggestion correct, would this be really a benefit? I also have to parse and interpret all responses in the respective data in my opinion.

Do i miss something?

francislavoie · March 5, 2024, 7:56am

I couldn’t tell you what the best approach is, just pointing to something in the same realm, i.e. code-gen or schema-gen based on Caddy’s JSON config.

Maybe it’s something you can make use of, maybe it’s not.

matt · March 5, 2024, 4:15pm

Would you like to propose a patch? That would speed things up a bit most likely.

flowbart · March 5, 2024, 8:49pm

Hi Matt,

I tried to get something working outside of caddy so far. Maybe there will be something to contribute when stuff is working.

This topic could be closed from my side so far.

Thank you for the fast feedback everybody.

BR
Florian

system · May 3, 2024, 8:39pm

This topic was automatically closed after 60 days. New replies are no longer allowed.