Whitehouse site blocks indexing of files about Iraq
Whitehouse.gov has a robots.txt file that keeps directories containing the word "iraq" from being indexed by search-engines.Sometime between April 2003 and late October, 2003, hundreds of instances of the term "iraq" were added to the whitehouse.gov robots.txt file.Link (Thanks, Kurt!)On a quick look, It appears that the google cached version of the robots.txt file very nearly could be created by searching for the string "text" in the April 16 file and replacing it with the string "iraq" then adding the newly changed files ("text" to "iraq") to the original file, keeping the lines with "text" in them as well. That's from a quick look so I'm not sure it would hold up, but it appears it could explain the bulk of the "iraq" appearances.
Share this post
Where not otherwise specified, this work is licensed under a Creative Commons License permitting non-commercial sharing with attribution. Boing Boing is a trademark of Happy Mutants LLC in the United States and other countries.






















