I am having this same issue. Did you ever find a solution?

Even the part of, if I am in the same building and on the same network as the Mail server it works fine. But when I am on some other...