# $Id: robots.txt,v 1.9.2.1 2008/12/10 20:12:19 goba Exp $## robots.txt## This file is to prevent the crawling and indexing of certain parts# of your site by web crawlers and spiders run by sites like Yahoo!# and Google. By telling these "robots" where not to go on your site,# you save bandwidth and server resources.## This file will be ignored unless it is at the root of your host:# Used:    http://example.com/robots.txt# Ignored: http://example.com/site/robots.txt## For more information about the robots.txt standard, see:# http://www.robotstxt.org/wc/robots.html## For syntax checking, see:# http://www.sxw.org.uk/computing/robots/check.html
User-agent: *
Disallow: /wp-
Disallow: /search
Disallow: /feed
Disallow: /comments/feed
Disallow: /feed/$
Disallow: /*/feed/$
Disallow: /*/feed/rss/$
Disallow: /*/trackback/$
Disallow: /*/*/feed/$
Disallow: /*/*/feed/rss/$
Disallow: /*/*/trackback/$
Disallow: /*/*/*/feed/$
Disallow: /*/*/*/feed/rss/$
Disallow: /*/*/*/trackback/$# DirectoriesDisallow: /includes/Disallow: /misc/Disallow: /modules/Disallow: /profiles/Disallow: /scripts/Disallow: /sites/Disallow: /themes/# FilesDisallow: /CHANGELOG.txtDisallow: /cron.phpDisallow: /INSTALL.mysql.txtDisallow: /INSTALL.pgsql.txtDisallow: /install.phpDisallow: /INSTALL.txtDisallow: /LICENSE.txtDisallow: /MAINTAINERS.txtDisallow: /update.phpDisallow: /UPGRADE.txtDisallow: /xmlrpc.php# Paths (clean URLs)Disallow: /admin/Disallow: /comment/reply/Disallow: /contact/Disallow: /logout/Disallow: /node/add/Disallow: /search/Disallow: /user/register/Disallow: /user/password/Disallow: /user/login/# Paths (no clean URLs)Disallow: /?q=admin/Disallow: /?q=comment/reply/Disallow: /?q=contact/Disallow: /?q=logout/Disallow: /?q=node/add/Disallow: /?q=search/Disallow: /?q=user/password/Disallow: /?q=user/register/Disallow: /?q=user/login/