[robots.txt] Disallow /azr94v2hh2lg/?

[Urk]

Een klant van mij heeft zijn site laten optimaliseren door Rexon, een SEO (Search Engine Optimizer), hij heeft een robots.txt geplaatst die het volgende zegt:

code:

User-agent: *
Disallow: /azr94v2hh2lg/

Als ik dit zoek via Google kom ik wel aardig wat hits tegen die hetzelfde doen. Echter kan ik niet vinden wat dit nou precies doet. Ergens lees ik wel:
"I won't pretend to be able to briefly or intelligently explain the rationale, but basically during "flux" periods when Google changes its formulas, it may fool itself into being disallowed even though the string allowing all, is present ... I see Googlebot/2.1, Googlebot-Image/1.0, and Mediapartners-Google/2.1 in my web stats at regular intervals now."

Maar daar word je ook geen wijs uit...
Kan iemand me vertellen wat er nou precies hier gebeurd en wat hiervan de bedoeling is?

[SinergyX]

UPDATE!!! (December 11, 2003)There is a major flaw in Google's robot when it comes to the robots.txt file so you MUST follow these instructions or your home page could be dropped from Google's index.The construction of the /robots.txt file is actually very simple. To create a /robots.txt file all you need is a simple ASCII text editor like Notepad (NOT Wordpad) and create a text file called robots.txt. Here are a couple of examples of what you might place in this file:To allow all robots complete access User-agent: *Disallow: /azr94v2hh2lg/Some say you can create an empty /robots.txt file instead of the above or to leave the "Disallow:" area empty, but if you do you could risk being dropped from Google. We have confirmed at least one member who has fallen victom to this so please follow these instructions.So why did we enter "/azr94v2hh2lg/" for the Disallow: area? This tells Google as well as any other robot type search engine to stay out of your "/azr94v2hh2lg/" directory. Of course you do not have this directory available and that is our point. You can replace the "/azr94v2hh2lg/" with any other directory name you wish just as long as the directory does not and will never be part of your Web site. You have been warned....If you would like a copy of the robots.txt above click here and save it to your local computer. Then upload this file to your Web server as instructed above.To exclude all robots from part of the server User-agent: *Disallow: /cgi-bin/Disallow: /tmp/Disallow: /private/Where does the /robots.txt file go?: The /robots.txt is to be placed in your root HTML document tree. Example;http://www.somesite.com/robots.txtThe filename robots.txt is case sensitive and MUST be in lowercase. If you name this file RObOTS.txt or anything other than robots.txt it will be incorrect.For more information on the Standard for Robots Exclusion (SRE) visit The Web Robots Pages (be careful of the Google flaw and do NOT follow their instructions on the empty Disallow: option in the robots.txt file).Robot Friendly HTML: All web pages are made up of HTML (HyperText Markup Language) code. Make sure the HTML code is "robot friendly" and the proper Tags are in the proper places. Most robot type search engines do not tolerate invalid HTML code, that is, if your code contains invalid HTML Tags, most search engines will not index your Web page or maybe even worse, not index you correctly.

eerste hit bekeken via google cache

[Urk]

Tnx, eigenlijk slaat het dus nergens op...? Dat was gewoon in bug in de Googlebot die nu -neem ik aan- allang gefixed is. Of zie ik dat verkeerd?

[BtM909]

Je kan eventueel ook even contact opnemen met google, aangezien het een google specifieke workaround is.