# This rule means it applies to all user-agents User-agent: * # Disallow all directories and files within Disallow: /contact/ Disallow: /reed/ Disallow: /rss/ Disallow: /wp-admin/ Disallow: /wp-includes/ Disallow: /files/ Disallow: /mediaplayer/ Disallow: /member/ # Disallow all monthly archive pages Disallow: /2006/0 Disallow: /2006/1 Disallow: /2007/0 Disallow: /2007/1 # The Googlebot is the main search bot for google User-agent: Googlebot # Disallow Google from parsing indididual post feeds and trackbacks.. Disallow: */feed/ Disallow: */trackback/ Disallow: /search Disallow: /feed Disallow: /comments/feed # weird MSIE thing that keeps hammering User-agent: Active Cache Request Disallow: *