I run Caddy with PHP over fastcgi very successfully on windows (despite @matt comments about people running webservers on windows 
)
A basic example is here
The important thing to do is startup the php fastcgi server as part of your caddyfile.
There is one other gotcha which is a php fastcgi issue not a Caddy issue and that is you must restart php fastcgi reguarily to prevent memory leaks. In effect this means restarting caddy if you are using the startup directive to start php fastcgi
The simplest thing to do is to write a script to do this for you. I will post a blog post with a script to do this soon.