External player?

[f1vefour]

XBMC "intelligently" throttles streaming based on the playing videos bitrate, sometimes XBMC gets it wrong or some other process needs bandwidth.

This is when the cache becomes important, with a larger cache XBMC has more time to recover from any mishap (dropped packets, retransmissions..etc). Otherwise a larger cache won't help if the issue is something else, or if you have no issue the cache size becomes unimportant.