My bad! I just found https://caddy.community/t/serving-tens-of-thousands-of-domains-over-https-with-caddy/11179
In my scenario, the use of On-demand TLS is needed. Also using ask directive is a must 
Thank you for your hard work, Caddy team. 
1 Like