I found that google calendar doesn't give the "couldn't fetch URL" error, if you substitute "http://" with "webcal://" and then remove the "?fmt=ics". I.e.
webcal://user%40server.com
asswd@my.zimbra.server.com/path/
I found that Zimbra insisted that the external user be an email address so the %40 work around also had to be used. The above works well on Apple iCal. Google accepts it, but now I'm waiting for my content. If Google accepts it, then that should make debugging easier. I'm using a valid cert on my Zimbra server.