WordPress MU

Donncha,

Just want to inform in WPMU 1.3.3, when you call robots.txt on browser, the url become to http://domain.com/robots.txt/ instead of http://domain.com/robots.txt

Just tested(firefox/IE7):
http://subliminalfuture.com/robots.txt
http://mathijsvanderkooi.start4all.com/robots.txt
http://blogivists.com/robots.txt
http://dailytestimony.net/robots.txt

Were is it being called from? I don't remember any bit in the code looking for that. The "In search engines" bit during signup puts in a meta line via wp_head.

edit: Doesn't show any trailing slash on our installs. Sounds more like your setup.

No, it adds the trailing slash on all the other example sites he listed below. IE; sites that are not paterick's and belong to others.

Still, the file is visible.

If you file a trac ticket, it'll get noticed faster than a forum post.

Hmm, that's strange. Still not showing up with our wpmu 1.3.3 installs.

Also they don't have it:

http://thunderlounge.com/robots.txt

Hmm, it does happen with edublogs though:

http://edublogs.org/robots.txt

Looks more like a setup issue though if it happens with some and not others.

Hmmm....

this could be one of those too simple to be true solutions (or too stupid to be true) but i do know that in the permalinks options for blogs the structures all have a trailing slash added.

Perhaps if you change this to a custom permalink structure without the trailing slash taht would solve your problem.. And explain why some work and others don't

Note: You can only change permalink structures on a blog by blog basis but since the file is in the root directory you should only have to update the main blog.

Hopefully this helps,

I have the same problem. I have a robots.txt empty file at the root of my install, http://musique.kune.fr/robots.txt , but when i add a http://musique.kune.fr/robots.txt/ (slash), it's giving me a
User-agent: *
Disallow:

It could not be a prob but i still have pages restricted that googlebot can't see, but i don't want it ...

Hmm when I add the slash, I get a 404, Mat_

i modify some files and it's seems to "work" now, i have the same thing if i had or not the /
its the wp-includes/functions.php to modify... But i still have pages who are not see by googlebot ...