Posts Tagged ‘robot.txt’
How to Avoid the Robot.txt Writer’s block: Get inspired by the big names!
Monday, February 22nd, 2010
![google-bot[12]](http://www.eustondigital.co.uk/blog/wp-content/uploads/2010/02/google-bot12.jpg "google-bot[12]")
When doing SEO changes for large scale companies, implementing a proper robot.txt is crucial.
I will not go back to the robot.txt definition bla, bla… millions have done that before us and here at Euston Digital we hate duplicate content… we prefer focusing on stunning link building services! 
No, instead, just a simple formula to use whenever the geek inside you has a Robot.txt Writer’s block!
So, type “inurl:robots.txt filetype:txt” and, ta-daa! See what the big names are doing.
You will find the robots.txt file from Google, Wikipedia, WebmasterWorld, the White House, Microsoft, W3.org, Facebook, IBM, Amazon, Ebay, New York Times, CNN, YouTube, etc.
Have Geek Fun and stayed tuned for new SEO tips soon.
Charly Wargnier
Tags: robot.txt, SEO
Posted in PPC | No Comments »
Categories
- And Finally (4)
- Conversion Optimisation (8)
- Display (4)
- Jobs (2)
- Mobile (4)
- Newsletters (2)
- PPC (28)
- Search Engines (17)
- SEO (19)
- Link Building (10)
- Social Media (13)
- Web Video (1)
Archives
-
-
