Obama's whitehouse.gov nukes 2400-line Bush robots.txt file

By at 3:09 pm Tuesday, Jan 20

Yesterday, the robots.txt file for whitehouse.gov had ca. 2400 lines worth of files and directories that search engines were not allowed to index. Today, the file is two lines long: "User-agent: *" and "Disallow: /includes/"
User-agent: *
Disallow: /cgi-bin
Disallow: /search
Disallow: /query.html
Disallow: /omb/search
Disallow: /omb/query.html
Disallow: /expectmore/search
Disallow: /expectmore/query.html
Disallow: /results/search
Disallow: /results/query.html
Disallow: /earmarks/search
Disallow: /earmarks/query.html
Disallow: /help
Disallow: /360pics/text
Disallow: /911/911day/text
Disallow: /911/heroes/text
The country's new robots.txt file

Read more    

Cory Doctorow

Upcoming appearances

* Feb 9, 2012, DeKalb, IL: Day of Doctorow, NIU
* Feb 10-12, 2012, Chicago, IL: Capricon 32
* Feb 13, 2012, Arlington, TX: UT Arlington College of Engineering Distinguished Speaker Series
* Feb 16, 2012, Victoria, BC: 13th Annual Privacy and Security Conference

Recent books:
* Context (essays)
* With a Little Help (short stories)
* For the Win (YA novel)
* Makers (adult novel)

Where not otherwise specified, this work is licensed under a Creative Commons License permitting non-commercial sharing with attribution. Boing Boing is a trademark of Happy Mutants LLC in the United States and other countries.