Every once in a while I end up setting up another reverse proxy/rewrite engine with nginx. In order to keep such configs easily portable you could do something like this:
location =/robots.txt {
add_header Content-Type text/plain;
return 200 "User-agent: *
Disallow: /
";
}
location =/robots.txt {
add_header Content-Type text/plain;
return 200 "User-agent: *
Disallow: /
";
}
location =/robots.txt {
add_header Content-Type text/plain;
return 200 "User-agent: *
Disallow: /
";
}
This eliminates the need to distribute an actual robots.txt file with the example above. If you intend to serve HTML instead, obviously change the MIME type to text/html as well.