The MU forums have moved to WordPress.org

Robots url should not has trailing slash (8 posts)

  1. mypatricks
    Member
    Posted 14 years ago #

    Donncha,

    Just want to inform in WPMU 1.3.3, when you call robots.txt on browser, the url become to http://domain.com/robots.txt/ instead of http://domain.com/robots.txt

    Just tested(firefox/IE7):
    http://subliminalfuture.com/robots.txt
    http://mathijsvanderkooi.start4all.com/robots.txt
    http://blogivists.com/robots.txt
    http://dailytestimony.net/robots.txt

  2. jackiedobson
    Member
    Posted 14 years ago #

    Were is it being called from? I don't remember any bit in the code looking for that. The "In search engines" bit during signup puts in a meta line via wp_head.

    edit: Doesn't show any trailing slash on our installs. Sounds more like your setup.

  3. andrea_r
    Moderator
    Posted 14 years ago #

    No, it adds the trailing slash on all the other example sites he listed below. IE; sites that are not paterick's and belong to others.

    Still, the file is visible.

    If you file a trac ticket, it'll get noticed faster than a forum post.

  4. jackiedobson
    Member
    Posted 14 years ago #

    Hmm, that's strange. Still not showing up with our wpmu 1.3.3 installs.

    Also they don't have it:

    http://thunderlounge.com/robots.txt

    Hmm, it does happen with edublogs though:

    http://edublogs.org/robots.txt

    Looks more like a setup issue though if it happens with some and not others.

    Hmmm....

  5. Radecki
    Member
    Posted 14 years ago #

    this could be one of those too simple to be true solutions (or too stupid to be true) but i do know that in the permalinks options for blogs the structures all have a trailing slash added.

    Perhaps if you change this to a custom permalink structure without the trailing slash taht would solve your problem.. And explain why some work and others don't

    Note: You can only change permalink structures on a blog by blog basis but since the file is in the root directory you should only have to update the main blog.

    Hopefully this helps,

  6. Mat_
    Member
    Posted 14 years ago #

    I have the same problem. I have a robots.txt empty file at the root of my install, http://musique.kune.fr/robots.txt , but when i add a http://musique.kune.fr/robots.txt/ (slash), it's giving me a
    User-agent: *
    Disallow:

    It could not be a prob but i still have pages restricted that googlebot can't see, but i don't want it ...

  7. jackiedobson
    Member
    Posted 14 years ago #

    Hmm when I add the slash, I get a 404, Mat_

  8. Mat_
    Member
    Posted 14 years ago #

    i modify some files and it's seems to "work" now, i have the same thing if i had or not the /
    its the wp-includes/functions.php to modify... But i still have pages who are not see by googlebot ...

About this Topic