Docker-based project server with automatic backups

Get answers to your questions about color grading, editing and finishing with DaVinci Resolve.
  • Author
  • Message
Offline

ematson5897

  • Posts: 8
  • Joined: Tue Oct 22, 2019 8:21 pm
  • Real Name: Elliot Matson

Docker-based project server with automatic backups

PostWed Nov 24, 2021 8:54 pm

After a long day of diving into Postgres and Project Servers, I'm writing up a guide on setting up a docker-compose stack for a Project server that runs automatic backups. I'm hoping this will be helpful to everyone using QNAP or Synology systems by making it easier to get a project server running.

The backups can be created at configurable intervals, and can be automatically pruned to keep x backups per day/week/month. The backup files are also compatible with Resolve's restore function.

A big part of this is using PostgreSQL 11.x.x, which is faster, significantly more secure, and not EOL, compared to the early 9.x.x versions that Resolve installs by default. On Windows, it's easy to uninstall PostgreSQL 9 and install 11, and when you run any database command Resolve will ask you to point it to the new PostgreSQL version. Is there a way to do that on a Mac? Without that, database tasks like backing up or optimizing fail.

Also, would anyone want to help test? I'm hoping to have a GitHub project up in the next few days
Offline

peterjackson

  • Posts: 1144
  • Joined: Sat Aug 18, 2018 7:12 pm
  • Real Name: Peter Jackson

Re: Docker-based project server with automatic backups

PostThu Nov 25, 2021 9:42 am

5950x, 3090, 128GB.
Offline

peterjackson

  • Posts: 1144
  • Joined: Sat Aug 18, 2018 7:12 pm
  • Real Name: Peter Jackson

Re: Docker-based project server with automatic backups

PostThu Nov 25, 2021 10:02 am

Apart from that I can hardly think of any worse place to run a PostgreSQL server than a qnap NAS.

Resolve workload is OLTP, lots of random IO that greatly benefits from living on a SSD and really gets trashed on parity based RAID levels like 5 and 6.

Then Qnaps are ultra low memory devices which again is the exact opposite
what an OLTP database wants.

A raspberry Pi with an SSD will likely outperform any such setup.

Postgres is extremely mature and longer around than most of you have internet access. There is really no need to come up with custom backup scripts. Just use industry standards and get on with life.
5950x, 3090, 128GB.
Offline

ematson5897

  • Posts: 8
  • Joined: Tue Oct 22, 2019 8:21 pm
  • Real Name: Elliot Matson

Re: Docker-based project server with automatic backups

PostThu Nov 25, 2021 2:58 pm

I'll have to look into pgbarman, looks pretty neat! The issue I see with this is that its archives are incompatible with Resolve's backup/restore functions, and my goal was to make things simple for smaller post houses that likely don't have IT staff.

I understand the concern about smaller/cheaper QNAP's with little RAM and no SSD's, but most people are running their OS and packages on SSD's these days, and anyone with a ZFS based box is likely going to have plenty of RAM. Really if it's a QNAP worth editing off of, it will likely have the resources to run a db. Again, my goal was simplicity, and setting up a Linux server from the ground up is not that.
Offline

ematson5897

  • Posts: 8
  • Joined: Tue Oct 22, 2019 8:21 pm
  • Real Name: Elliot Matson

Re: Docker-based project server with automatic backups

PostSun Nov 28, 2021 7:07 pm

Alright, here is the GitHub link if anyone is interested. If anyone has a Synology NAS and can test installation or contribute an installation guide that would be very helpful

https://github.com/ematson5897/Docker-Davinci-Resolve-Project-Server
Offline

JDWidell

  • Posts: 4
  • Joined: Sun Sep 22, 2019 8:41 am
  • Real Name: Daniel Widell

Re: Docker-based project server with automatic backups

PostThu Jul 28, 2022 1:37 pm

ematson5897 wrote:Alright, here is the GitHub link if anyone is interested. If anyone has a Synology NAS and can test installation or contribute an installation guide that would be very helpful

https://github.com/ematson5897/Docker-Davinci-Resolve-Project-Server


Maybe a little late to the party, but I just updated to resolve 18 now that it got out of beta and gave it a try on my QNAP NAS. It works beautifully! Of course I saved my old QNAP container as a backup, but compared to the hassle of setting up the NAS the first time this makes the whole process A LOT easier. Good work (I guess you're the main author)!
Offline

ematson5897

  • Posts: 8
  • Joined: Tue Oct 22, 2019 8:21 pm
  • Real Name: Elliot Matson

Re: Docker-based project server with automatic backups

PostFri Jul 29, 2022 2:43 pm

JDWidell wrote:
ematson5897 wrote:Alright, here is the GitHub link if anyone is interested. If anyone has a Synology NAS and can test installation or contribute an installation guide that would be very helpful

https://github.com/ematson5897/Docker-Davinci-Resolve-Project-Server


Maybe a little late to the party, but I just updated to resolve 18 now that it got out of beta and gave it a try on my QNAP NAS. It works beautifully! Of course I saved my old QNAP container as a backup, but compared to the hassle of setting up the NAS the first time this makes the whole process A LOT easier. Good work (I guess you're the main author)!


Glad to hear it!
Offline

jdmdmm

  • Posts: 2
  • Joined: Wed Jan 18, 2023 9:26 am
  • Real Name: Jonathan de Montfort

Re: Docker-based project server with automatic backups

PostWed Jan 18, 2023 9:57 am

Good morning,

I'm very interested in this for deploying the project server to a QNAP NAS on Docker using the Container Station - thankyou for putting this together and saving us all some time!

I noticed that you mention to put the db files on an SSD and I wondered why?
We're using a RAID 5 configuration with groups of 4 (providing approx 18Gb/s througput) with 2Tb SSD Caching - would that suffice?

Regards,
Jon
Offline

ematson5897

  • Posts: 8
  • Joined: Tue Oct 22, 2019 8:21 pm
  • Real Name: Elliot Matson

Re: Docker-based project server with automatic backups

PostWed Jan 18, 2023 7:41 pm

jdmdmm wrote:Good morning,

I'm very interested in this for deploying the project server to a QNAP NAS on Docker using the Container Station - thankyou for putting this together and saving us all some time!

I noticed that you mention to put the db files on an SSD and I wondered why?
We're using a RAID 5 configuration with groups of 4 (providing approx 18Gb/s througput) with 2Tb SSD Caching - would that suffice?

Regards,
Jon

Are you on QTS, or QuTS Hero?
Offline

Domingo Olmo Martin

  • Posts: 69
  • Joined: Mon Aug 27, 2012 3:09 pm

Re: Docker-based project server with automatic backups

PostWed Jan 18, 2023 7:57 pm

I use an official Docker PostgreSQL database on Raspberry Pi 4 that backs up all Resolve databases every day, deletes the 30+1 day backup so I still have 30 days of backup, archives the last day of the month backup.
It work with a combination of cron that calls some shell scripts (maybe I should have used Python instead)
It works very effectively and has saved my ass several times.

I did some testing with the PostgreSQL database on a Synology NAS, but it was too stressful whenever an update appeared for the NAS, and there were some incidents that could have been very problematic.

My advice is that NAS are meant for putting files on and that's it.
Domingo Olmo Martin
Author - Director
Co-founder of Studio Barsoom
Offline

jdmdmm

  • Posts: 2
  • Joined: Wed Jan 18, 2023 9:26 am
  • Real Name: Jonathan de Montfort

Re: Docker-based project server with automatic backups

PostThu Jan 19, 2023 10:41 am

ematson5897 wrote:
jdmdmm wrote:Good morning,

I'm very interested in this for deploying the project server to a QNAP NAS on Docker using the Container Station - thankyou for putting this together and saving us all some time!

I noticed that you mention to put the db files on an SSD and I wondered why?
We're using a RAID 5 configuration with groups of 4 (providing approx 18Gb/s througput) with 2Tb SSD Caching - would that suffice?

Regards,
Jon

Are you on QTS, or QuTS Hero?


We're on QTS 5.0.1.2248

Return to DaVinci Resolve

Who is online

Users browsing this forum: okiewardoyo, tarloff and 153 guests