Ok, so this is a problem I never had myself, but friends of mine has had similar problems, the SBS team have found a workaround that works most of the time. You can read about it at “The Official SBS Blog”

http://blogs.technet.com/sbs/archive/2009/01/28/slow-connectivity-for-outlook-anywhere-and-sites-that-use-the-sbs-web-applications-app-pool.aspx

Mikael Nystrom – TrueSec
MCT, MVP Windows Server – Setup/Deployment