# /robots.txt file for http://webcrawler.com/
# mail sanjeev@cdc-creative.co.uk for constructive criticism

User-agent: *
Disallow: /links/
Disallow: /error_docs/
Disallow: /css/
Disallow: /Client_Side_Detection/
Sitemap: http://www.cdc-creative.co.uk/sitemap.xml