Hi Ashish,
This is a common misunderstanding. When Caddy is starting, there are no sites to take down, because Caddy was not running before. If you stopped the server before, then you took the sites down yourself. On the other hand, if you use signal USR1 to reload an updated Caddyfile while Caddy is running, Caddy will not stop serving the old sites if a new one fails to come online.
Additionally, this is the same as other web servers like nginx. If it can’t resolve the domain you tell it to bind to, it can’t start. Definitely not a design flaw.
Once your DNS has been configured correctly, you can use Caddy (or other web servers) to serve your site.