Yes, I'm pretty sure you do have to keep render switched on in order to use cache for delivery. The reason I believe this is because as soon as you turn of render cache, it unlinks your clips to the cache, and even if you turn it right back on, it has to completely re-render everything from scratch.
On a side note, I find it particularly annoying to disable nodes in color page to check before/after, and having the cache disconnected and needing complete re-render.