How to Avoid the Robot.txt Writer’s block: Get inspired by the big names!
![google-bot[12]](http://www.eustondigital.co.uk/blog/wp-content/uploads/2010/02/google-bot12.jpg "google-bot[12]")
When doing SEO changes for large scale companies, implementing a proper robot.txt is crucial.
I will not go back to the robot.txt definition bla, bla… millions have done that before us and here at Euston Digital we hate duplicate content… we prefer focusing on stunning link building services! 
No, instead, just a simple formula to use whenever the geek inside you has a Robot.txt Writer’s block!
So, type “inurl:robots.txt filetype:txt” and, ta-daa! See what the big names are doing.
You will find the robots.txt file from Google, Wikipedia, WebmasterWorld, the White House, Microsoft, W3.org, Facebook, IBM, Amazon, Ebay, New York Times, CNN, YouTube, etc.
Have Geek Fun and stayed tuned for new SEO tips soon.
Charly Wargnier
This entry was posted on Monday, February 22nd, 2010 at 3:38 pm and is filed under PPC. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.
Leave a Reply