Okay I think I’ve figured this out. Documenting in case it’s helpful to some poor future soul.
Once I followed the instructions in this post and added the below snippet to my wp-config.php, everything started working.
if (!empty($_SERVER['HTTP_X_FORWARDED_PROTO']) && $_SERVER['HTTP_X_FORWARDED_PROTO'] === 'https') {
$_SERVER['HTTPS'] = 'on';
}
With this snippet it no longer matters whether the “Site Address” is set to http or https, it just works.
It seems like this shouldn’t be required and this should be fixable by making sure that the headers are getting passed through correctly. But I couldn’t figure out any combination of headers that made WP behave as I expected.