# robots.txt - 20070522 User-agent: * Crawl-delay: 3 # Directories Disallow: /modules/ Disallow: /sites/ Disallow: /themes/ Disallow: /scripts/ # Paths (clean URLs) Disallow: /admin/ Disallow: /aggregator/ Disallow: /comment/reply/ Disallow: /contact/ Disallow: /logout/ Disallow: /node/add/ Disallow: /user/register/ Disallow: /user/password/ Disallow: /user/login/ # specific Disallow: /cart/ Disallow: /admin/ Disallow: /vergroting