Automatic HTTPS for Route53 Multi-Profile Support

Hi all, I’m currently running Caddy from a docker image (latest), and alpine builder (latest)…

I was able to get Automatic HTTPS working (thru DNS Validation) by building caddy with route53 dns module and adding the default “Profile” in the AWS Credentials configuration in the ~/.aws/credentials path as in:

[default]
aws_access_key_id = <YOUR_DEFAULT_ACCESS_KEY_ID>
aws_secret_access_key = <YOUR_DEFAULT_SECRET_ACCESS_KEY>

Everything was working fine until the time I needed to add another “Profile” from another AWS account as in:

[development]
aws_access_key_id = <YOUR_DEFAULT_ACCESS_KEY_ID>
aws_secret_access_key = <YOUR_DEFAULT_SECRET_ACCESS_KEY>

[production]
aws_access_key_id = <YOUR_DEFAULT_ACCESS_KEY_ID>
aws_secret_access_key = <YOUR_DEFAULT_SECRET_ACCESS_KEY>

And now I don’t know how to specify in the virtual-host section of the caddyfile about which aws profile/account it should use to connect for route53 dns validation…


Here’s my current caddyfile:

dev.example.com:443 {
  bind {$ADDRESS}
  header X-Robots-Tag noindex
  reverse_proxy /* dev:80 {
      header_up Host "dev.example.com"
  }
  tls {
      dns route53 # from the development aws account
  }
}
prod.example.com:443 {
  bind {$ADDRESS}
  header X-Robots-Tag noindex
  reverse_proxy /* dev:80 {
      header_up Host "prod.example.com"
  }
  tls {
      dns route53 # from the production aws account
  }
}

In contrast to the cloudflare DNS module, I can specify the API key in the caddyfile itself so no worries on how many cloudflare accounts I use it with as in:

dev.example.com:443 {
  bind {$ADDRESS}
  header X-Robots-Tag noindex
  reverse_proxy /* dev:80 {
      header_up Host "dev.example.com"
  }
  tls {
      dns cloudflare "redacted-key"
  }
}

Is there somehow a way to specify the AWS credentials profile that I want to use for DNS validation?

I think you should open an issue on the repo to ask for help on this:

can you please try setting the AWS_PROFILE environment variable to the profile name you want to use in your container? That should do it!

1 Like

@danlsgiga

Tried it already, but it will only limit me to use one Profile at a time so it will only let me use Automatic HTTPS for domains thats registered in route53 in that aws account.

I need to use 2 different profiles to make it work for Automatic HTTPS for two domains. The production and development domains belong to two different aws account, so different access keys/id.

Or am i missing something here? :sweat_smile::sweat_smile::sweat_smile:

I’ve never heard of any application requiring to use multiple AWS profiles concurrently… even the aws cli does not support it… you need to specify what profile to use for every call. If you need to work across different AWS accounts / roles, you need to do that at the IAM level assigning a role that gives you access to all resources you need.

Hi @danlsgiga

I’ve never heard of any application requiring to use multiple AWS profiles concurrently

For most application, a single profile is enough to get things done. However, I strongly believe that there will come a time where people can’t deal things at the IAM level due to some restrictions–this scenario provides the need to have multi-profile support for automatic HTTPS in the route53 module.


you need to specify what profile to use for every call.

I agree with that. However, the current way of specifying a profile is Global (means you can only use 1 profile for all virtualhosts)… For me I think that method is very limiting… So I suggest that we add a new argument in the tls directive where people can specify a profile per virtual-hosts,… if there’s no profile specified in the virtual host, then it can fall back to a global/default profile.. Example caddyfile:

assets.staging-site.com:443 {
  bind {$ADDRESS}
  reverse_proxy /* web-server:80 {
      header_up Host "assets.staging-site.com"
  }
  tls {
      dns route53 {
        max_retries 10
        aws_profile "staging-profile" # New argument for multi-profile support
      }
  }
}

assets.production-site.com:443 {
  bind {$ADDRESS}
  reverse_proxy /* web-server:80 {
      header_up Host "assets.production-site.com"
  }
  tls {
      dns route53 {
        max_retries 10
        aws_profile "production-profile" # New argument for multi-profile support
      }
  }
}

even the aws cli does not support it

AWS-cli is very different from AWS-GO-SDK…
Please check this link to see the SDK support of case-based configuration for aws-go-sdk:
https://docs.aws.amazon.com/sdk-for-go/api/aws/session/#hdr-Creating_Sessions


If you need to work across different AWS accounts / roles, you need to do that at the IAM level assigning a role that gives you access to all resources you need.

As I said, handling things at the IAM level may not be an option for everyone.

Dear @francislavoie @danlsgiga, I have forked the route53 module and its libdns dependency…

I’ve made some changes and added the feature that I was needing. Please see the following links below to and check if I am up to your standards as the maintainer of the original repo. hehehehe! :crossed_fingers:

https://github.com/PhoenixPeca/route53/blob/v1.0.29/route53.go#L57-L64

https://github.com/PhoenixPeca/route53-1/blob/v1.0.21/provider.go#L14

https://github.com/PhoenixPeca/route53-1/blob/v1.0.21/client.go#L23-L34

let me know if you agree with these changes so that I can make a PR into the original repos. :smiley:
you can change the code however you deem fit. :smiley:
Thanks!


PS: I haven’t coded in Go-lang before, so it took me almost 5 hours to research, implement, and test this feature. LOL.

1 Like

Hi all, I have opened a pull request in the concerned repositories to add this feature. Let me know if there are any issues or anything… :smiley:

https://github.com/libdns/route53/pull/1

https://github.com/caddy-dns/route53/pull/5


Please review… :smiley:

1 Like

Ok, this makes sense to me after considering your requirements.

1 Like

Hi @danlsgiga, cool! I will check @matt 's review later today, and make some changes.

The current code works already, though still need improvements to comply on the code standards.
Eitherway, still need you to double check, since I’m pretty new to golang, im more of a PHP guy hahha!. :joy: If you can join in the discussion or make a recommendation over at github, please do!

Really appreciate all you guys are doing to help me achieve this goal. I’d rather spend hours working on a fix for this than to spend time every 2 months trying to renew ssl. :laughing:

This is fine, thanks for contributing… I think this will benefit more people for sure!

1 Like

Can we get a new release of this Route53 plugin with these changes? The docs reflect this new feature but are out of sync with actual releases (doing a build with xcaddy to include the Route53 module currently breaks because it does not recognize the aws_profile attribute.

See this message for more details: Renewing a Wildcard certificate - #5 by harrisreynolds

cc @danlsgiga

–harris

Hi @harrisreynolds, apologies for the delay… The Added multi-profile support #1 PR works already as per my testing… However, I haven’t been able to find time to test it with “EC2 Instance profiles” as per @danlsgiga ‘s request… :’(

The past few days have been rough for me (at work). I’ll try to test this later tonight to and push this through…


The docs reflect this new feature but are out of sync with actual releases.

I thought that this won’t be documented until all PR’s have been merged and a proper release was done…
I mean, I thought people will all refer to this documentation?


Edit: Should anyone want to help me do the testing for “EC2 Instance Profiles”, please do… All help is really appreciated.

1 Like

This topic was automatically closed after 30 days. New replies are no longer allowed.