Update on my troubleshooting.
Good news: I no longer have the issue.
bad news: I don't think that I did anything to fix it.
I went back to a previous project I've just finished that was the same size and opened that. It had no issues scrubbing or playing the timeline. the RAM usage stayed at about 33%.
I went back to the project with issues and it was also at around 33% till I selected a 4k clip and tried to play that. My memory spiked again and it led to a crash.
I then opened the project again, but I did not select a 4k clip until all clips were rendered. It now scrubs and plays back as expected.
I did go into the preferences and changed the Memory and GPU setting to use just 10 GB of system memory instead of the max it was set at. I'm thinking that a background application was also using memory while Resolve was trying to render my playback files. I had a Chrome window open during one crash (I normally have nothing else open beside Resolve) and that crashed when Resolve crashed, so it looks like windows did not like how much memory was being used. Antimalware Service Executable is always high up in my memory usage in the Task Manager, so maybe it was doing a scan or an update or something else in the background.
I'll keep an eye on the crashing to see if I just need to pause occasionally when adding a large number of clips to the timeline to let the render cache catch up.