Replacement for DPX format?

Get answers to your questions about color grading, editing and finishing with DaVinci Resolve.
  • Author
  • Message
Offline

Nick Verlinden

  • Posts: 166
  • Joined: Wed Aug 12, 2015 11:08 am

Replacement for DPX format?

PostFri Jan 24, 2020 10:31 am

Hi,

I manage the production facilities for a client who does film restoration. Right now the entire workflow is built around DPX. From scanning to mastering, everything is DPX. We also archive all of the scans and masters on LTO tape, and have gathered a petabyte of data.

I was wondering if we could change our workflow to something more modern, that utilises some mathematical compression to save space. DPX is entirely uncompressed, and it just seems like something we would do in the old days because CPU power was the bottleneck and DPX was universally adopted across software packages. Now the bottleneck has become storage and bandwidth. Our SAN environment is ageing and will soon need replacement, and the budgets are tight. I started looking at alternatives, such as a NAS. Unfortunately the data troughput of a NAS is not as high as that of a SAN, and doubt it cloud handle more than one 4k DPX stream. 10gbe also has not enough bandwidth to transport even a single 4k DPX stream.

So long question short: is there something else we can use other than DPX that is a lot smaller in size, but retains the original quality?

I come looking for some advice, all is welcome!

Kind regards,
Nick
macOS 10.13.2 High Sierra
Core i9 7920X, 32GB RAM
GTX 1080 Ti 11GB
Blackmagic DeckLink Mini Monitor 4K
Blackmagic URSA Mini Pro
Blackmagic Pocket Cinema Camera
Blackmagic DaVinci Resolve Studio 15
Offline

Tero Ahlfors

  • Posts: 640
  • Joined: Fri Apr 03, 2015 3:02 pm

Re: Replacement for DPX format?

PostFri Jan 24, 2020 10:59 am

EXR with compression?
Offline

Hendrik Proosa

  • Posts: 3015
  • Joined: Wed Aug 22, 2012 6:53 am
  • Location: Estonia

Re: Replacement for DPX format?

PostFri Jan 24, 2020 11:19 am

Tero Ahlfors wrote:EXR with compression?

This. EXR is the best there is, different lossless compression methods (zip 1 and 16 scanline, Piz) and visually lossless DWA. Plus very flexible metadata and channel management.
I do stuff.
Offline

Nick Verlinden

  • Posts: 166
  • Joined: Wed Aug 12, 2015 11:08 am

Re: Replacement for DPX format?

PostFri Jan 24, 2020 5:32 pm

Ah yes, I thought about EXR. Would there be any argument today to opt for DPX instead of compressed EXR?
macOS 10.13.2 High Sierra
Core i9 7920X, 32GB RAM
GTX 1080 Ti 11GB
Blackmagic DeckLink Mini Monitor 4K
Blackmagic URSA Mini Pro
Blackmagic Pocket Cinema Camera
Blackmagic DaVinci Resolve Studio 15
Offline

Jim Simon

  • Posts: 29806
  • Joined: Fri Dec 23, 2016 1:47 am

Re: Replacement for DPX format?

PostFri Jan 24, 2020 6:01 pm

Nick Verlinden wrote:Would there be any argument today to opt for DPX instead of compressed EXR?


Compatibility. You may still find DPX works in more places than EXR does, so check every part of your pipeline before switching.
My Biases:

You NEED training.
You NEED a desktop.
You NEED a calibrated (non-computer) display.
Offline
User avatar

carlomacchiavello

  • Posts: 2587
  • Joined: Tue Aug 28, 2012 6:04 pm
  • Location: italy

Re: Replacement for DPX format?

PostSun Jan 26, 2020 7:19 am

Jim Simon wrote:
Nick Verlinden wrote:Would there be any argument today to opt for DPX instead of compressed EXR?


Compatibility. You may still find DPX works in more places than EXR does, so check every part of your pipeline before switching.


If some your tools not support openExr, it’s time to change. OpenExr is a vfx standard from since twenty years. I start to use it near 2001 thanks to ability to have multi layer, channel, 3d data compressed lossless to manage better workflow from software 3d to postproduction.
Just check if all your software read correctly color profile, someone also after 21 years of born do mistake on color profile embedded of openExr


Inviato dal mio iPhone utilizzando Tapatalk
Offline
User avatar

Marc Wielage

  • Posts: 10901
  • Joined: Fri Oct 18, 2013 2:46 am
  • Location: Hollywood, USA

Re: Replacement for DPX format?

PostSun Jan 26, 2020 9:32 am

Nick Verlinden wrote:I was wondering if we could change our workflow to something more modern, that utilises some mathematical compression to save space. DPX is entirely uncompressed, and it just seems like something we would do in the old days because CPU power was the bottleneck and DPX was universally adopted across software packages. Now the bottleneck has become storage and bandwidth. Our SAN environment is aging and will soon need replacement, and the budgets are tight.

I quite a bit of film restoration work here in LA, and one of our main clients does everything in ProRes 444 or 444XQ, which I would argue is "virtually lossless" (12-bit and at source resolution). You can argue that it's proprietary, but it pretty much plays on anything we throw at it and the film scans hold up very well, all the way through HDR/Dolby Vision.

The advantage of DPX is that it's open source and a good legacy format; the disadvantage is that it takes up tons of space: 2 hours of 12-bit 2K DPX is 2.25TB, while 2 hours of 12-bit 2K ProRes 444XQ is 461.6GB. I think all the major film restoration software (MTI, Diamond, Nucoda, Resolve, etc.) work fine with ProRes in and out.
marc wielage, csi • VP/color & workflow • chroma | hollywood
Offline

Hendrik Proosa

  • Posts: 3015
  • Joined: Wed Aug 22, 2012 6:53 am
  • Location: Estonia

Re: Replacement for DPX format?

PostSun Jan 26, 2020 9:53 am

OpenEXR is also open source. Depending on workflow working with image sequences can be essential or not. When it is not, any modern intra-only codec works just as good prores, dnx, cineform etc.
I do stuff.
Offline

Nick Verlinden

  • Posts: 166
  • Joined: Wed Aug 12, 2015 11:08 am

Re: Replacement for DPX format?

PostTue Jan 28, 2020 2:59 pm

Thanks Marc for the info. I appreciate it.
While I do agree with you, I'm unable to sell the idea of adopting ProRes in an end to end workflow to upper management.

I'm testing some EXR workflows now with PIZ compression. In first tests I seem to gain about 42% of space.
We have a Northlight scanner, which is only able to scan DPX files, but I can convert these to EXR using ImageMagick faster than the scanner works.

I'm going to try and implement EXR once we have a replacement for our ageing Revival machines.

Thanks all for pitching in!
macOS 10.13.2 High Sierra
Core i9 7920X, 32GB RAM
GTX 1080 Ti 11GB
Blackmagic DeckLink Mini Monitor 4K
Blackmagic URSA Mini Pro
Blackmagic Pocket Cinema Camera
Blackmagic DaVinci Resolve Studio 15

Return to DaVinci Resolve

Who is online

Users browsing this forum: AdamPWSmith, Baidu [Spider], peeceful, SteveW, VMFXBV and 168 guests