Page 1 of 2 12 LastLast
Results 1 to 10 of 15

Thread: Results total & paging information

  1. #1
    furball4 is offline Intermediate Member
    Join Date
    Oct 2006
    Posts
    23
    Rep Power
    8

    Default Results total & paging information

    When in an area where more items exist than are shown, I see that I can page forward (e.g., the Accounts list in the admin tool). But I can't see the total number of items that I am paging through, or the number of pages. In some areas it shows the current numbers, such as 50-75, but still no totals (e.g., Email folders). Am I missing something? It is not, practically speaking, a very big issue. But I was expecting to find it somewhere.

  2. #2
    jdell is offline Project Contributor
    Join Date
    Jul 2006
    Location
    Reno, NV, USA
    Posts
    203
    Rep Power
    8

    Default

    Quote Originally Posted by furball4
    When in an area where more items exist than are shown, I see that I can page forward (e.g., the Accounts list in the admin tool). But I can't see the total number of items that I am paging through, or the number of pages. In some areas it shows the current numbers, such as 50-75, but still no totals (e.g., Email folders). Am I missing something? It is not, practically speaking, a very big issue. But I was expecting to find it somewhere.
    I was actually wondering the same thing just a few days ago, and forgot about it until I just read your post here.

    bug 9976 http://bugzilla.zimbra.com/show_bug.cgi?id=9976 is for contacts, but it is the same idea. The last comment on the bug also mentions emails not displaying totals.

    Vote for the bug. I did.

  3. #3
    cdamon is offline Zimbra Employee
    Join Date
    Sep 2005
    Posts
    53
    Rep Power
    9

    Default

    In general, we don't show the total because we don't know it. For the sake of efficiency, our server performs searches with a LIMIT clause. For example, if you search your mail for 'xml', you will only get one page's worth of results. If you click the arrow to get the next page, a new search with a different LIMIT clause is performed.

    There will be exception - for example, we load all of the user's contacts on startup since we need them for tooltips and autocomplete. So bug 9976 is tractable. I'd recommend filing a different bug for seeing email totals (eg folder sizes), though it may never get fixed.

    -Conrad
    Bugzilla - Wiki - Downloads - Before posting... Search!

  4. #4
    jdell is offline Project Contributor
    Join Date
    Jul 2006
    Location
    Reno, NV, USA
    Posts
    203
    Rep Power
    8

    Default

    Quote Originally Posted by cdamon
    In general, we don't show the total because we don't know it. For the sake of efficiency, our server performs searches with a LIMIT clause. For example, if you search your mail for 'xml', you will only get one page's worth of results. If you click the arrow to get the next page, a new search with a different LIMIT clause is performed.
    -Conrad
    Ok, well that kinda stinks. But I'd rather have Zimbra perform well...

    Digging around, there is a MySQL SQL_CALC_FOUND_ROWS meta-column you can throw in your LIMIT queries, that allows you to get total results.

    This article explains it, although some replies say performance still isn't as good as basic LIMIT search, I suspect that if you have proper indexing, it could be very fast. I haven't tried this myself on other mysql projects.

    http://www.arraystudio.com/as-worksh...ing-limit.html

    Either way, it is an interesting article.

  5. #5
    furball4 is offline Intermediate Member
    Join Date
    Oct 2006
    Posts
    23
    Rep Power
    8

    Default

    Yeah, I completely understand the pagination issue with regards to performance. It would take an extra query to find the total number of results.

    One option would be to make people mouse over something or otherwise request the total before making the query. That way it would happen only rarely, and then it would most likely be worth the expense. Seems a bit heavy-handed for a minor feature though. jdell's idea sounds very interesting too.

  6. #6
    tim brennan is offline Zimbra Employee
    Join Date
    Sep 2005
    Posts
    93
    Rep Power
    9

    Default Search is not just from MySQL

    Quote Originally Posted by jdell
    Digging around, there is a MySQL SQL_CALC_FOUND_ROWS meta-column you can throw in your LIMIT queries, that allows you to get total results.
    Running a search is NOT just as simple as doing a MySQL query. Not all of the data is stored in MySQL: MySQL's full-text capabilities are poor, and in some cases (e.g. shared contacts) the data is not on your local server. Take a relatively simple search: "tag:foo subject:john" -- the tag data is stored in MySQL, but the subject data is stored in Lucene. So executing that query becomes a matter of doing a JOIN of results from the two different data stores.

    If you think it through for a minute, you'll realize that there's no way to calculate the number of results of a join without actually doing it -- even if you know there are 1000 rows in the "tag:foo" part, and 10000 in the "subject:john" part, that only tells you that the result set falls between 0-1000 hits....you have to actually fetch all the rows and do the join to find out. Since it is common for a search part to match almost every row (-in:trash) you'll realize that many many searches would end up touching basically every row in the user's mailbox: computationally infeasible in a shared server environment.

    Even the large search engines have trouble with this, and they have (in some ways) a simpler problem since they don't have the kind of dynamic data we have. When you run a Google search it might say "Result 1-10 out of about 9,123,000" ....but that number is really just a guess: a statistical estimate of what the join should give you, it is not an actual counting of the results by any means.

    Anyway, to make a long story short, while it might be possible for us to give result sizes in some cases, in most cases we can't practically do it. We've talked about having the server count up to a certain point (say, count up to 500 results and then say "more than 500" beyond that) ....but so far we've opted for the simpler and more efficent mode of not giving a count. Since people almost never look beyond the first page of a result set, it makes more sense to get that first page back as fast as possible instead of scanning ahead to do a count.
    Bugzilla - Wiki - Downloads - Before posting... Search!

  7. #7
    jdell is offline Project Contributor
    Join Date
    Jul 2006
    Location
    Reno, NV, USA
    Posts
    203
    Rep Power
    8

    Default

    Quote Originally Posted by tim brennan
    Anyway, to make a long story short, while it might be possible for us to give result sizes in some cases, in most cases we can't practically do it. We've talked about having the server count up to a certain point (say, count up to 500 results and then say "more than 500" beyond that) ....but so far we've opted for the simpler and more efficent mode of not giving a count. Since people almost never look beyond the first page of a result set, it makes more sense to get that first page back as fast as possible instead of scanning ahead to do a count.
    I hear you, and I now understand this better. Thanks for that :-)

    My 2 cents...

    Just because you can't solve it for all cases doesn't mean the problem is intractable. How about breaking it down into separate problems, like counting just for email (and not in searches).

    There are times when I'd really like to know how many items are in a folder (particularly emails). Currently, I fire up Thunderbird/IMAP for that. Saying people don't look past the first page of results is mostly true, but for some cases, that is definitely not true and not knowing how many is quite annoying.

    As far as email, I'd be happy just to get a context menu item, say 'properties' on a folder that tells you totals such as # of emails, perhaps space taken, etc.

    Obviously for large installs, getting the counts is not computationally feasible as you say, but how about SMB's. Why does Zimbra have to be a one-size-fits-all deal?

    I'm responsible for 3 NE sales that were < 100 users each. These installs get their own server and have tons of disk space and CPU per user compared to big installs.

    Why not do something with COS to enable/disable?

    Finally, the stop at 500 and just say 500+ is a great idea that again ought to be controlled through COS (configured depending on the size of the install).

    Thanks again for the detailed explanation.

    Regards,
    John

  8. #8
    dkarp is offline Zimbra Employee
    Join Date
    Aug 2005
    Posts
    1,433
    Rep Power
    11

    Default Slightly different issue.

    Quote Originally Posted by jdell
    There are times when I'd really like to know how many items are in a folder (particularly emails). Currently, I fire up Thunderbird/IMAP for that. Saying people don't look past the first page of results is mostly true, but for some cases, that is definitely not true and not knowing how many is quite annoying.

    As far as email, I'd be happy just to get a context menu item, say 'properties' on a folder that tells you totals such as # of emails, perhaps space taken, etc.
    I think this is a slightly different bug. Making the number of items in a folder visible (somehow) is bug 9976. Please vote for it!
    Bugzilla - Wiki - Downloads - Before posting... Search!

  9. #9
    jdell is offline Project Contributor
    Join Date
    Jul 2006
    Location
    Reno, NV, USA
    Posts
    203
    Rep Power
    8

    Default

    Quote Originally Posted by dkarp
    I think this is a slightly different bug. Making the number of items in a folder visible (somehow) is bug 9976. Please vote for it!
    I already voted for that bug a while ago

    Anyway, I understand better how this works, and why it is such a hard problem(s).

    I humbly predict this isn't the last time you will have to explain to someone why you can't do this

    Regards,
    John

  10. #10
    jbwiv is offline Senior Member
    Join Date
    Feb 2007
    Posts
    58
    Rep Power
    8

    Default

    What I'm curious about...if it's so different, then why is everyone else (GMail, Microsoft, etc) able to do it? Seriously guys....even if it's a performance hit it should be an option. I'd rather run a query twice than have an unknown results set. Additionally, this doesn't just apply to searches...I don't even get a full page count in my Inbox...

Page 1 of 2 12 LastLast

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Similar Threads

  1. Replies: 8
    Last Post: 04-21-2007, 10:29 PM
  2. AD GAL: successful, but empty, test results
    By tmadden in forum Administrators
    Replies: 0
    Last Post: 10-02-2006, 03:36 PM

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •