Results 1 to 9 of 9

Thread: [SOLVED] robots.txt?

  1. #1
    Rich Graves is offline Outstanding Member
    Join Date
    Jan 2007
    Location
    Minnesota
    Posts
    718
    Rep Power
    9

    Default [SOLVED] robots.txt?

    I'd like to discourage automated pounding of REST URLs, googling of unintentionally public wikis, google-hacking in case I'm a day late in applying a critical security patch, etc. [How] can I add a restrictive root-level robots.txt?

    Yes, I'm well aware that this isn't a real security measure, but it helps, especially against casual google-hacking.

  2. #2
    sturgis is offline Senior Member
    Join Date
    Jan 2007
    Location
    France
    Posts
    71
    Rep Power
    8

    Default

    Quote Originally Posted by Rich Graves View Post
    I'd like to discourage automated pounding of REST URLs, googling of unintentionally public wikis, google-hacking in case I'm a day late in applying a critical security patch, etc. [How] can I add a restrictive root-level robots.txt?

    Yes, I'm well aware that this isn't a real security measure, but it helps, especially against casual google-hacking.
    Did you find the way?
    I don't like google to index my zimbra...

  3. #3
    jholder's Avatar
    jholder is offline Former Zimbran
    Join Date
    Oct 2005
    Location
    Thatcher, AZ
    Posts
    5,606
    Rep Power
    20

    Default

    Robots.txt doesn't really apply to dynamic pages.
    As for the "unintentionally public wikis", google's only going to find them if there's a link to them somewhere.

  4. #4
    Rich Graves is offline Outstanding Member
    Join Date
    Jan 2007
    Location
    Minnesota
    Posts
    718
    Rep Power
    9

    Default

    Robots.txt doesn't really apply to dynamic pages.
    Huh? All search engines follow https links with query parameters nowadays (Google was the first to do so). But if you tell robots.txt to Disallow * from /, Google won't index the site.

    We just need to know how to drop a robots.txt into the virtual web root. I'm sure I could figure it out eventually, but I don't (currently) speak Java or Tomcat and am not that convinced that I need to speak the latter if you're switching to Jetty in just a few months.

  5. #5
    tikal is offline Senior Member
    Join Date
    Mar 2008
    Location
    Seattle
    Posts
    52
    Rep Power
    7

    Default

    So what do I need to do to create a robots.txt file for all the different domains I'm using to access the server? Can I create one robots file for all the domains, or does a different one need to be created for each domain. I'm only using six domain names to access the server, so it would not be tough to create them all individually. Various bots are are showing up in my log files looking a robots.txt file and trying to index things. I do not want this. Can someone point me in the right direction or give me instructions?

  6. #6
    Rich Graves is offline Outstanding Member
    Join Date
    Jan 2007
    Location
    Minnesota
    Posts
    718
    Rep Power
    9

    Default

    I haven't done this, but remain interested in doing so.

  7. #7
    shawnlhood is offline Active Member
    Join Date
    Sep 2007
    Posts
    48
    Rep Power
    7

    Default

    *bump*

    I don't like my webmail login page being indexed.

  8. #8
    Rich Graves is offline Outstanding Member
    Join Date
    Jan 2007
    Location
    Minnesota
    Posts
    718
    Rep Power
    9

    Default

    You can add <META NAME="ROBOTS" CONTENT="NOINDEX, NOFOLLOW"> to the <head> section of /opt/zimbra/jetty/webapps/zimbra/public/login.jsp; zmmailboxd stop; flush the /opt/zimbra/jetty/work/zimbra/jsp/org/apache/jsp/public_/ directory; zmmailboxd start.

    This would have to be done again after every minor software update.

    If you're really still running 4.5.6, that's /opt/zimbra/tomcat. If you're not, update your forum profile.

    I'd still like to be able to drop in a robots.txt to deal with the case of spiders starting somewhere other than root (public wikis, calendars, etc).

  9. #9
    tikal is offline Senior Member
    Join Date
    Mar 2008
    Location
    Seattle
    Posts
    52
    Rep Power
    7

    Default

    I feel that a robots.txt file preconfigured with "no indexing" options should come as a default part of ZCS. Or at least, the aforementioned meta tags should be included as a default. Although I'm sure some people would not like this feature, I feel pretty confident that most would really appreciate this. I really am not looking forward to having to manually fix this after every ZCS update.
    Last edited by tikal; 09-18-2008 at 01:29 PM.

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •