ProRes RAW for UMP

The place for questions about shooting with Blackmagic Cameras.
  • Author
  • Message
Offline

Adam Silver

  • Posts: 301
  • Joined: Wed Dec 14, 2016 4:12 pm

ProRes RAW for UMP

PostThu Apr 05, 2018 3:59 pm

Apple is releasing ProRes RAW. Sounds interesting. I wonder if it could be added to the UMP.

https://www.apple.com/newsroom/2018/04/ ... aptioning/

Adam
Offline

Gavin_c_clark

  • Posts: 299
  • Joined: Thu Apr 16, 2015 3:51 pm

Re: ProRes RAW for UMP

PostThu Apr 05, 2018 4:07 pm

Looks interesting but skimming through the white paper I think it’ll be chunkier than compressed cine dng?

Also if it only works in fcpx and only on Mac it might be a bit limited...

Fortunately it’s a free update so I’ll see for myself on April 9!
Offline
User avatar

Donnell Henry

  • Posts: 1109
  • Joined: Wed Dec 25, 2013 9:04 pm
  • Location: Brooklyn ny

Re: ProRes RAW for UMP

PostThu Apr 05, 2018 4:14 pm

Wow that’s amazing indeed ..if you can pull the same information from prores raw as you do with Raw or raw 3:1 files from Cdng’s what a game changer that will be for a more streamlined editing and color correcting process. It even says that it takes up less space than prores 4444. It could also mean that on black magic cameras we can probably get higher frames rates with the prores raw option. The article mentioned atamos shogun and DJI inspire drones will be getting it in an update. I’m 200% sure we will be getting it in our cameras as well. Thanks for posting this.
GODS CREATE
Offline

Denny Smith

  • Posts: 13131
  • Joined: Thu Aug 01, 2013 4:19 pm
  • Location: USA, Northern Calif.

Re: ProRes RAW for UMP

PostThu Apr 05, 2018 4:23 pm

Perhaps this is the mysterious Raw released on the Micro Studio Camera? :roll:
Cheers
Denny Smith
SHA Productions
Offline

FrankApollonio

  • Posts: 215
  • Joined: Mon Aug 12, 2013 5:13 pm

Re: ProRes RAW for UMP

PostThu Apr 05, 2018 7:58 pm

REALLY REALLY REALLY WANT THIS ON URSA MINIS .... DROP THAT **** INSTANTLYYYY BMD ... and please drop a new camera at NAB ...and please ship next week. thanks!
Offline
User avatar

carlomacchiavello

  • Posts: 2576
  • Joined: Tue Aug 28, 2012 6:04 pm
  • Location: italy

Re: ProRes RAW for UMP

PostThu Apr 05, 2018 8:03 pm

2018 prores raw, that work only in finalcutProX, not compatible for other application.
2005 Cineform inc release a version of CIneform compatible with every application Os codec driven with raw and active metadata decoding and realtime color, rawdecoding, stereo settings, and more...

instead to add proresRaw, another locked and brand linked codec, Os limited, i hope bmd add CineformRaw to their products, an opensource - completely open and documented well developed codec.
i think fee to apple will be heavier than 20$ per device asked for raw support from Cineform raw.
Offline

Nate Porter

  • Posts: 211
  • Joined: Wed May 01, 2013 8:22 am

Re: ProRes RAW for UMP

PostThu Apr 05, 2018 9:23 pm

This is day one. Who knows BMD could have support for it soon. I'd love it in the ursa minis, cinemadng while great for being open source, isn't as easy of a workflow so projects that I need done quickly I skip RAW. This makes me wonder does apple have the leverage still in the creative field that it once did when it made prores as ubiquitous. I'd much rather have BMD implement Cineform RAW but that's not happening soon.
Offline

Ryan Hamblin

  • Posts: 218
  • Joined: Sat Apr 18, 2015 10:04 pm
  • Location: LA/Nashville, TN

Re: ProRes RAW for UMP

PostThu Apr 05, 2018 10:44 pm

Our company is pretty much a dedicated FCPx post company. Having prores Raw on the 4.6k and the pro would be a game changer for us and our workflow.
www.brainstem.tv
www.ryanhamblin.com
Offline

Timothy Cook

  • Posts: 241
  • Joined: Sat Mar 01, 2014 1:10 am
  • Location: Ft. Worth, Texas

Re: ProRes RAW for UMP

PostThu Apr 05, 2018 11:18 pm

Yeah, this is going to be big for Mac users like myself.
Watching FCPX blaze through Prores files is insane, now having the flexibility of RAW to go with that.... :shock:

Looks like Atomos is going to be the first recorders to support it. RAW directly from camera to recorder processed in Prores RAW.
I'm sure BMD has already know about it and is prepping Resolve for it. Which by the way already does a pretty good job with Non-Raw Prores on a Mac.

Matt has a really good write up on it here. https://www.newsshooter.com/2018/04/06/ ... w-is-here/

Realtime skimming and playback of 8K Prores RAW on an average MacBook looks like it's practically here.
Good time to be alive, lol.
Vimeo.com/dropbars
Offline
User avatar

Jamie LeJeune

  • Posts: 2012
  • Joined: Tue Feb 26, 2013 4:33 am
  • Location: San Francisco

Re: ProRes RAW for UMP

PostThu Apr 05, 2018 11:59 pm

Ryan Hamblin wrote:Our company is pretty much a dedicated FCPx post company. Having prores Raw on the 4.6k and the pro would be a game changer for us and our workflow.

You might want to take a look at Apple's white paper on it:
https://www.apple.com/final-cut-pro/docs/Apple_ProRes_RAW_White_Paper.pdf

According to the white paper, FCPX debayers the ProRes raw files (presumably based on camera metadata) to either a user specified log color space, or an unspecified wide gamut color space if no log curve is chosen. There are no controls in the FCPX inspector tab for adjusting temp, tint, or ISO metadata BEFORE the debayer. The white paper says users can apply color adjustments in FCPX using the color wheels, but those adjustments are applied downstream AFTER the debayer (otherwise things like the masking controls would not be possible). So, as far as workflow inside FCPX is concerned, there is little functional difference between recording ProRes RAW versus 12 bit ProRes444 log files. The whole benefit to a raw file is to be able to change temp, tint and ISO metadata in post prior to debayer, but that's not how these files will work in FCPX according to Apple.

It's conceivable that at a later date, Apple ProRes raw files will be readable by Resolve with proper metadata controls in Resolve's raw tab. But if your goal is to stay entirely in FCPX, it won't give you the control over metadata prior to debayer that we currently have in Resolve with other raw formats.
The glass half full way of looking at it is that if you record to 12 bit ProRes 444 log files on your UMP today, you've already got everything that ProRes raw will someday give you in FCPX ;)
www.cinedocs.com
http://www.imdb.com/name/nm4601572/
Offline

Timothy Cook

  • Posts: 241
  • Joined: Sat Mar 01, 2014 1:10 am
  • Location: Ft. Worth, Texas

Re: ProRes RAW for UMP

PostFri Apr 06, 2018 12:18 am

Jamie, thanks for digging through the white paper.

I agree with just using 4444, a less compressed codec, rather than Prores RAW (IF) FCPX truly doesn't support the temp. and ISO advantages of RAW.

And yes, hopefully Resolve implements this in the new update.
Vimeo.com/dropbars
Offline
User avatar

carlomacchiavello

  • Posts: 2576
  • Joined: Tue Aug 28, 2012 6:04 pm
  • Location: italy

Re: ProRes RAW for UMP

PostFri Apr 06, 2018 6:15 am

Dji add.it to x7s camera with prores and dng to offer better space quality ratio.
I hope there is space to add it to other cameras, not only bmd.
But my hope is for cineform raw which a production proven workflow from more than 10 years and is immediately compatible with all software. Active metadata idea give you ability to implement all features easily and faster, and it was first codec to use gpu acceleration to manage color stereo etc.

Inviato dal mio E6653 utilizzando Tapatalk
Offline

Michael McCaffrey

  • Posts: 798
  • Joined: Tue Apr 14, 2015 1:30 am

Re: ProRes RAW for UMP

PostFri Apr 06, 2018 6:15 am

FrankApollonio wrote:REALLY REALLY REALLY WANT THIS ON URSA MINIS .... DROP THAT **** INSTANTLYYYY BMD ... and please drop a new camera at NAB ...and please ship next week. thanks!


YeS! ProRes RAW on the UM4.6K!! I hate cinemadng. It’s so bloated and processor intensive I don’t even shoot raw on my raw camera because it just takes WAY too much data and it won’t even playback on the computer in real-time with a 2013 12-core MacPro and a GTX 1080 Ti eGPU. Can’t stand cinemadng. I’ve been waiting for BMD to release their own codec. Maybe ProRes RAW will fit the bill.
Configuration:
Resolve Studio (Always the Latest)
Windows 11 Pro Workstation
32 Core AMD Ryzen Threadripper 256GB RAM
RTX 4090
RTX 3090
100G NAS
(1) 32" Ultra-Wide Display, (1) 4K 27” Display
Offline
User avatar

Dmytro Shijan

  • Posts: 1760
  • Joined: Wed Sep 17, 2014 7:15 pm
  • Location: UA

Re: ProRes RAW for UMP

PostFri Apr 06, 2018 7:32 am

Michael McCaffrey wrote:
FrankApollonio wrote:REALLY REALLY REALLY WANT THIS ON URSA MINIS .... DROP THAT **** INSTANTLYYYY BMD ... and please drop a new camera at NAB ...and please ship next week. thanks!


YeS! ProRes RAW on the UM4.6K!! I hate cinemadng. It’s so bloated and processor intensive I don’t even shoot raw on my raw camera because it just takes WAY too much data and it won’t even playback on the computer in real-time with a 2013 12-core MacPro and a GTX 1080 Ti eGPU. Can’t stand cinemadng. I’ve been waiting for BMD to release their own codec. Maybe ProRes RAW will fit the bill.


From white paper pdf that ProRes RAW is same size as dng 3:1 and 4:1. It will not helps to save storage space.
Many tech people on forums report that that 3:1 and 4:1 DNG uses same compression method as ProRes.
In white paper pdf there is no any comparison of ProRes RAW with 4:1 and 3:1 compressed DNG. For unknown reason they just ignore that format.
As we know 4:1 and 3:1 compressed DNG introduced by Blackmagic and still not supported in most applications.
I think ProRes RAW may be just some kind of evolution of cinema DNG format for video. I think it may be commonly accepted if allow faster playback and encoding than compressed DNG.
BMMCC/BMMSC Rigs Collection https://bmmccrigs.tumblr.com
My custom made accessories for BMMCC/BMMSC https://lavky.com/radioproektor/
Offline
User avatar

David Chapman

  • Posts: 461
  • Joined: Thu Nov 08, 2012 11:05 pm
  • Location: Dallas, TX

Re: ProRes RAW for UMP

PostFri Apr 06, 2018 10:47 am

I’ve been reading up on ProRes RAW and one person in particular is saying that ProRes RAW and ProRes RAW HQ are intended to replace ProRes 444 and ProRes HQ for some users. It’s not necessarily a RAW replacement? Just sensor data vs an rgb conversion.

I’ll be at NAB and all of the Apple events to find out more. But if FCPX won’t allow for ISO/temp changes and 4:1 is comparable, then 4:1 might be better for some cases? If BMD add these new formats as options.
David Chapman
Just another creative dude with a camera.
Offline

John Paines

  • Posts: 5777
  • Joined: Tue Jul 28, 2015 4:04 pm

Re: ProRes RAW for UMP

PostFri Apr 06, 2018 1:58 pm

Dmitry Shijan wrote:From white paper pdf that ProRes RAW is same size as dng 3:1 and 4:1. It will not helps to save storage space.


Which is why all this enthusiasm is puzzling.... Apple compares performance of this new format to Redcode raw and Canon raw light, but comparisons to Cineform raw and CDNG are missing. Remains to be seen, no?
Offline

JasonFinnigan

  • Posts: 63
  • Joined: Fri Feb 03, 2017 2:58 pm

Re: ProRes RAW for UMP

PostFri Apr 06, 2018 2:58 pm

John Paines wrote:
Dmitry Shijan wrote:From white paper pdf that ProRes RAW is same size as dng 3:1 and 4:1. It will not helps to save storage space.


Which is why all this enthusiasm is puzzling.... Apple compares performance of this new format to Redcode raw and Canon raw light, but comparisons to Cineform raw and CDNG are missing. Remains to be seen, no?


Don't you know, anything with the word Apple before it is automatically better :)
Offline

Jim Giberti

  • Posts: 275
  • Joined: Tue Mar 03, 2015 2:03 am

Re: ProRes RAW for UMP

PostFri Apr 06, 2018 4:04 pm

JasonFinnigan wrote:Don't you know, anything with the word Apple before it is automatically better


So you prefer apple turds.
Oh, I get it, you were being condescending.
Offline
User avatar

carlomacchiavello

  • Posts: 2576
  • Joined: Tue Aug 28, 2012 6:04 pm
  • Location: italy

Re: ProRes RAW for UMP

PostFri Apr 06, 2018 4:25 pm

i reed white paper and i found some weak points ProresRaw against CineformRaw
- CineformRaw could be do in camera (kineinfinity, SI2k, CInerecorder) or you can convert arriraw, redcodec, dng, most of dslr raw to cineformraw without problem vs ProresRaw where you need a camera or a recorder with Proresraw to have this codec.

- CFraw you can use CF Raw in every software that read codec from os in windows, mac, and today that is opensource will be also easely under linux, actually ProresRaw is read only by FinalcutProX, also a simple dailty is a pain... i hope an update for qt update to read it, but every daily tools be obsolete.

- CF active metadata database driven tool vs simple passive decoding mean different and complex managment in the workflow.

- CFraw is opensource, ProresRaw is patent Apple, and you know, also if word Apple could do all better, you pay a lot this, in first you cannot add to all software that you want... At today prores is only under MacOs, and few software use really the power of Prores (tons of feature wasted and forgetted from NLE / PostPro software developer.
Offline

Wayne Steven

  • Posts: 3362
  • Joined: Thu Aug 01, 2013 3:58 am
  • Location: Earth

Re: ProRes RAW for UMP

PostFri Apr 06, 2018 4:31 pm

Denny Smith wrote:Perhaps this is the mysterious Raw released on the Micro Studio Camera? :roll:
Cheers


If only. But that camera needs a sensor upgrade. The series it uses is old, and they have just announced latest ones.
aIf you are not truthfully progressive, maybe you shouldn't say anything
bTruthful side topics in-line with or related to, the discussion accepted
cOften people deceive themselves so much they do not understand, even when the truth is explained to them
Offline

Howard Roll

  • Posts: 2491
  • Joined: Fri Jun 03, 2016 7:50 am

Re: ProRes RAW for UMP

PostFri Apr 06, 2018 4:41 pm

Well that only took forever. Maybe Apple decided to give a squirt about the pro side of the business again. I've been saying for years that any 4:2:2 or even 4:4:4 sampling of a single chip sensor is a waste of data. Apple and BM have been playing nice for years, I would expect BM implementation across their product line to be right around the corner.
Offline

Wayne Steven

  • Posts: 3362
  • Joined: Thu Aug 01, 2013 3:58 am
  • Location: Earth

Re: ProRes RAW for UMP

PostFri Apr 06, 2018 4:42 pm

An issue with cineform raw, which cineform try to mitigate, is video processing systems on computers were optimised for compression systems of the type ProRes falls under, and not wavelet. Gives a huge advantage.

Now, the interesting part here. Is while it may seem to be nearly as much data as prores 4:4:4, RAW is three times smaller anyway, so you might get 1/3rd the compression ratio of cineform 4:4:4. Which is good.
Last edited by Wayne Steven on Fri Apr 13, 2018 1:29 am, edited 3 times in total.
aIf you are not truthfully progressive, maybe you shouldn't say anything
bTruthful side topics in-line with or related to, the discussion accepted
cOften people deceive themselves so much they do not understand, even when the truth is explained to them
Offline

Wayne Steven

  • Posts: 3362
  • Joined: Thu Aug 01, 2013 3:58 am
  • Location: Earth

Re: ProRes RAW for UMP

PostFri Apr 06, 2018 4:46 pm

Yeah, a bit of a waste, but Bayer itself is an issue. A true three colour pixel sensor (given sigma X3 being one) can produce very nice 4:4:4 which gives s you better results than Bayer. Canon used to just use 1080p on a 4k sensor, to increase pixel quality, I expect.
aIf you are not truthfully progressive, maybe you shouldn't say anything
bTruthful side topics in-line with or related to, the discussion accepted
cOften people deceive themselves so much they do not understand, even when the truth is explained to them
Offline

John Brawley

  • Posts: 4264
  • Joined: Tue Aug 21, 2012 7:57 am
  • Location: Los Angeles California

Re: ProRes RAW for UMP

PostFri Apr 06, 2018 4:52 pm

ProRes RAW is a great thing.

It gives you RAW controls inside of ProRes. I believe it’s been leaked or announced elsewhere that Resolve supports this already. It’s not just FCPx.

ProRes implementations aren’t closed to reading ProRes files so I imagine it’s just up the the ProRes app developers to write their code and you’ll see other platforms supporting this.

Cineform is dead.

They wanted too much money to licence. Now they’re owned by GoPro and have gone exactly nowhere.

It doens’t matter if they’re smarter or better. They don’t have any market support.

ProRes support is hardware assisted and it’s built into the OS and Hardware of each Apple product and they make damn sure it works very very reliably when originating to it in camera.

You will see this in cameras very soon. Think of almost any camera that shoots ProRes now, and it could potentially do ProRes RAW. It will only be up to the will of the manufacturer to implement it.

Like ProRes or not, this is big news and I suspect it will be rapidly adopted. I’d guess that the MAJORITY of the worlds high end production isn’t RAW, it’s ProRes, for many many reasons that have nothing to do with IQ and everything to do with workflow. Now you get both.

JB
John Brawley ACS
Cinematographer
Currently - Los Angeles
Offline
User avatar

Dmytro Shijan

  • Posts: 1760
  • Joined: Wed Sep 17, 2014 7:15 pm
  • Location: UA

Re: ProRes RAW for UMP

PostFri Apr 06, 2018 5:22 pm

Yea, Cineform was just not too constant and not too stable. Too many rebranding, too many different codec versions and variations, too often radical changes in software. It just should be simpler and more user friendly.
Also RED patented wavelet in-camera compression concept, so they destroyed Cineform RAW In-camera compression future.
BMMCC/BMMSC Rigs Collection https://bmmccrigs.tumblr.com
My custom made accessories for BMMCC/BMMSC https://lavky.com/radioproektor/
Offline
User avatar

rick.lang

  • Posts: 17156
  • Joined: Wed Aug 22, 2012 5:41 pm
  • Location: Victoria BC Canada

ProRes RAW for UMP

PostFri Apr 06, 2018 5:27 pm

Dmitry Shijan wrote:YeS! ProRes RAW on the UM4.6K!!
...
In white paper pdf there is no any comparison of ProRes RAW with 4:1 and 3:1 compressed DNG. For unknown reason they just ignore that format.
As we know 4:1 and 3:1 compressed DNG introduced by Blackmagic and still not supported in most applications.
I think ProRes RAW may be just some kind of evolution of cinema DNG format for video. I think it may be commonly accepted if allow faster playback and encoding than compressed DNG.


The CinemaDNG I thought was BMD’s innovation by using the Adobe DNG spec and applying it to a folder representing a video clip. The 3:1 and 4:1 flavours were BMD’s totally. So we shouldn’t be surprised or upset if Apple FCP X doesn’t support compressed CinemaDNG. The ProRes raw and ProRes raw HQ belong to Apple as we know from the new white paper describing this new flavour.

The write ups from NewsShooter and particularly Cinema5D seemed to just regurgitate whatever Apple wanted printed about the new codec. The Cinema5D article seemed to call it remarkably different as if it skipped the debayer step, but as Jamie and Matt Allard indicated, it’s still raw that needs to be turned into RGB pixels for example.

And being raw, we are assured you’ll be able to adjust ISO, temperature, and tint so provision for that must surely be coming to FCP X.

I’d like to see it in Resolve 15 and in a few days we shall know. I do believe Resolve will do an excellent job accommodating ProRes raw/HQ with the Colour raw tab, but no idea of BMD’s implementation timeline.
Rick Lang
Offline

JasonFinnigan

  • Posts: 63
  • Joined: Fri Feb 03, 2017 2:58 pm

Re: ProRes RAW for UMP

PostFri Apr 06, 2018 5:38 pm

John Brawley wrote:Cineform is dead.

They wanted too much money to licence. Now they’re owned by GoPro and have gone exactly nowhere.

It doens’t matter if they’re smarter or better. They don’t have any market support.


There's a reason it started closed source, then went open. There was no money to be made on it. It's not that great of a codec. If ProRes supports actual RAW controls (which the whitepaper didn't seem to say for sure that it would) It will be great as it will be much easier to get support in other Apps than CDNG even, Heck Avid might even support it :)
Offline

JasonFinnigan

  • Posts: 63
  • Joined: Fri Feb 03, 2017 2:58 pm

Re: ProRes RAW for UMP

PostFri Apr 06, 2018 5:39 pm

Jim Giberti wrote:
JasonFinnigan wrote:Don't you know, anything with the word Apple before it is automatically better


So you prefer apple turds.
Oh, I get it, you were being condescending.


I was just making a joke. Not taking Sides I use all vendors products.
Offline

JasonFinnigan

  • Posts: 63
  • Joined: Fri Feb 03, 2017 2:58 pm

Re: ProRes RAW for UMP

PostFri Apr 06, 2018 5:41 pm

rick.lang wrote: The CinemaDNG I thought was BMD’s innovation by using the Adobe DNG spec and applying it to a folder representing a video clip.



BMD had nothing to do with it. Adobe made it all open source. https://www.adobe.com/devnet/cinemadng.html. Now the compressed versions are theirs fully
Offline

John Brawley

  • Posts: 4264
  • Joined: Tue Aug 21, 2012 7:57 am
  • Location: Los Angeles California

Re: ProRes RAW for UMP

PostFri Apr 06, 2018 5:48 pm

JasonFinnigan wrote:
rick.lang wrote: The CinemaDNG I thought was BMD’s innovation by using the Adobe DNG spec and applying it to a folder representing a video clip.



BMD had nothing to do with it. Adobe made it all open source. https://www.adobe.com/devnet/cinemadng.html. Now the compressed versions are theirs fully


They created the compressed versions, but it’s still cDNG so it’s still open to anyone that chooses to support them.

JB
John Brawley ACS
Cinematographer
Currently - Los Angeles
Offline

Jim Giberti

  • Posts: 275
  • Joined: Tue Mar 03, 2015 2:03 am

Re: ProRes RAW for UMP

PostFri Apr 06, 2018 8:23 pm

John Brawley wrote:ProRes RAW is a great thing.
I’d guess that the MAJORITY of the worlds high end production isn’t RAW, it’s ProRes, for many many reasons that have nothing to do with IQ and everything to do with workflow. Now you get both.JB


We're pretty psyched in our small creative shop JB.
As much as I love the RAW control and mojo we're shooting and editing everything in ProRes.
This is such a great thing if it's going to be a genuine best of both worlds - and it sure sounds like it's going to be.
Offline
User avatar

rick.lang

  • Posts: 17156
  • Joined: Wed Aug 22, 2012 5:41 pm
  • Location: Victoria BC Canada

Re: ProRes RAW for UMP

PostFri Apr 06, 2018 10:33 pm

Thanks for the corrections Jason and John.



Sent from my iPhone using Tapatalk
Rick Lang
Offline
User avatar

carlomacchiavello

  • Posts: 2576
  • Joined: Tue Aug 28, 2012 6:04 pm
  • Location: italy

Re: ProRes RAW for UMP

PostFri Apr 06, 2018 10:41 pm

Dmitry Shijan wrote:Yea, Cineform was just not too constant and not too stable. Too many rebranding, too many different codec versions and variations, too often radical changes in software. It just should be simpler and more user friendly.
Also RED patented wavelet in-camera compression concept, so they destroyed Cineform RAW In-camera compression future.


I use it from years without big problems, but i talk about commercial version. Like John told some post over, CineformRaw is near to be dead, from when Gopro buy it and close develop of commercial version. Now is opensource and if someone want to implement software is free, hardware cost 20$ for camera, not so expansive for ecosystem.
Wavelet could be patented by red, but newman use wavelet compression it before Red in first cineform incarnation in 2001, and in camera like raw wavelet 2005, released in 2006 with PsTechnique SI2k, first Red camera come out was 2007, be cause they announce at nab 2006, but first prototype used by Peter Jackson for some test arrive in 2007.

Anyway, if they add in camera both codec i will be happy, more options...
but... actually from what i know users of Ursa still attend dnhr... grayed menu from a bit of time...
i Hope Grant give us many great news
Offline

JasonFinnigan

  • Posts: 63
  • Joined: Fri Feb 03, 2017 2:58 pm

Re: ProRes RAW for UMP

PostFri Apr 06, 2018 11:23 pm

With Apple planning to drop Intel Chips for their computers in favor of using Arm based chips. I wonder how ProRes support will be for Non-MacOS devices in a few years?
Offline

Wayne Steven

  • Posts: 3362
  • Joined: Thu Aug 01, 2013 3:58 am
  • Location: Earth

Re: ProRes RAW for UMP

PostSat Apr 07, 2018 12:10 am

John Brawley wrote:ProRes RAW is a great thing.

It gives you RAW controls inside of ProRes. I believe it’s been leaked or announced elsewhere that Resolve supports this already. It’s not just FCPx.

ProRes implementations aren’t closed to reading ProRes files so I imagine it’s just up the the ProRes app developers to write their code and you’ll see other platforms supporting this.

Cineform is dead.

They wanted too much money to licence. Now they’re owned by GoPro and have gone exactly nowhere.

It doens’t matter if they’re smarter or better. They don’t have any market support.

ProRes support is hardware assisted and it’s built into the OS and Hardware of each Apple product and they make damn sure it works very very reliably when originating to it in camera.

You will see this in cameras very soon. Think of almost any camera that shoots ProRes now, and it could potentially do ProRes RAW. It will only be up to the will of the manufacturer to implement it.

Like ProRes or not, this is big news and I suspect it will be rapidly adopted. I’d guess that the MAJORITY of the worlds high end production isn’t RAW, it’s ProRes, for many many reasons that have nothing to do with IQ and everything to do with workflow. Now you get both.

JB



I don't know, the license they offered me was pretty good, now free plus the raw license for the chesp $20 seperate standard based on it (why the old one is open sourced). Why wouldn't companies just license the version of the new standard instead. But what John said about ProRes raw, I essentially agree with. Hardware support in existing models might or might not cover it, and it might not be the greatest, is compressed, which apparently every body around here except for me hates, but it has better hardware support, unlike wavelet.

The market has been dominated and denied certain things due to the will of some players. Why did GoPro buy cineform, it was rumoured they might do a cinema camera, and suddenly stalled as something else happened. The sensor access has been another issue. But cineform has been used in a few cameras. So, not unsuccessful and extremely good for the day. Some guy came up with the idea of changing picture by meta data before the final implementation, and they implemented it.

But so many suspicious moves and explicable sudden U-turns in the industry which directly kept lower cost better equipment out of the hands of professional and private consumers, favouring competitors who also had lower end markets selling rubbish at high priices, over the years, but no US senate enquiry about the billions of dollars being hemorrhaged out of the US this way, maybe each year. Which is a loss of maybe over a billion dollars in tax revenue. Somebody should pay.
Last edited by Wayne Steven on Fri Apr 13, 2018 1:28 am, edited 1 time in total.
aIf you are not truthfully progressive, maybe you shouldn't say anything
bTruthful side topics in-line with or related to, the discussion accepted
cOften people deceive themselves so much they do not understand, even when the truth is explained to them
Offline

Andrew Kolakowski

  • Posts: 9206
  • Joined: Tue Sep 11, 2012 10:20 am
  • Location: Poland

Re: ProRes RAW for UMP

PostSat Apr 07, 2018 12:36 pm

Dmitry Shijan wrote:Yea, Cineform was just not too constant and not too stable. Too many rebranding, too many different codec versions and variations, too often radical changes in software. It just should be simpler and more user friendly.
Also RED patented wavelet in-camera compression concept, so they destroyed Cineform RAW In-camera compression future.


I don't think these are the reasons (there were never many versions- core was always the same).
Main reason is that Cineform was developed by 1 guy who had no power to push it forward.
Cineform is a very plain form a wavelet based encoding (which is its best strength), but in the same time it's few times faster to decode than eg. RED. I bet you many people would prefer slightly less efficient Cineform than RED for the price of so much easier decoding. RED is just an encrypted JPEG2000 which is a nightmare for decoding.
One of the reason why Cineform was never "properly" implemented in camera is the fact that it doesn't have constant bitrate mode (as it was made as high end archiving/finishing codec), which is a problem for camera manufactures. Bitrate can jump from frame to frame a lot which makes it difficult to implement. ProRes is also VBR, but it allows only for e.g. 10% peaks from target bitrate which is way easier to "handle" by developers.
David many times said that Cineform in terms of chip processing power is not more demanding than ProRes or DNxHR. Problem was/is related to the way how Cineform operates in terms of bitrate- it has "very VBR" based nature, which is not camera implementation friendly. It would require new constant compression ratio mode, which was never added to SDK.
Last edited by Andrew Kolakowski on Sat Apr 07, 2018 1:24 pm, edited 2 times in total.
Offline

Andrew Kolakowski

  • Posts: 9206
  • Joined: Tue Sep 11, 2012 10:20 am
  • Location: Poland

Re: ProRes RAW for UMP

PostSat Apr 07, 2018 12:53 pm

JasonFinnigan wrote:
John Brawley wrote:Cineform is dead.

They wanted too much money to licence. Now they’re owned by GoPro and have gone exactly nowhere.

It doens’t matter if they’re smarter or better. They don’t have any market support.


There's a reason it started closed source, then went open. There was no money to be made on it. It's not that great of a codec. If ProRes supports actual RAW controls (which the whitepaper didn't seem to say for sure that it would) It will be great as it will be much easier to get support in other Apps than CDNG even, Heck Avid might even support it :)


It's as great as ProRes or DNxHD etc. (actually even more interesting). It's very simple in the core which is exactly what makes is "great". Whole set of features around it with active metadata etc is still very unique and way ahead of anything else out there. Lack of support from big name company is the key reason for its current state- otherwise it would be mainstream by now.
ProRes is nothing special (although well designed from the ground) at all and it's so popular only because it's Apple developed.
Last edited by Andrew Kolakowski on Sat Apr 07, 2018 1:24 pm, edited 1 time in total.
Offline

Andrew Kolakowski

  • Posts: 9206
  • Joined: Tue Sep 11, 2012 10:20 am
  • Location: Poland

Re: ProRes RAW for UMP

PostSat Apr 07, 2018 1:02 pm

Dmitry Shijan wrote:Yea, Cineform was just not too constant and not too stable. Too many rebranding, too many different codec versions and variations, too often radical changes in software. It just should be simpler and more user friendly.
Also RED patented wavelet in-camera compression concept, so they destroyed Cineform RAW In-camera compression future.


RED is just an encrypted JPEG2000 and it's purely used by RED itself. What patents does RED hold there related to R3D? I would not link Cineform lack of popularity with RED at all.
Offline

Andrew Kolakowski

  • Posts: 9206
  • Joined: Tue Sep 11, 2012 10:20 am
  • Location: Poland

Re: ProRes RAW for UMP

PostSat Apr 07, 2018 1:14 pm

David Chapman wrote:I’ve been reading up on ProRes RAW and one person in particular is saying that ProRes RAW and ProRes RAW HQ are intended to replace ProRes 444 and ProRes HQ for some users. It’s not necessarily a RAW replacement? Just sensor data vs an rgb conversion.

I’ll be at NAB and all of the Apple events to find out more. But if FCPX won’t allow for ISO/temp changes and 4:1 is comparable, then 4:1 might be better for some cases? If BMD add these new formats as options.


Encoding engine will be most likely very similar or exactly the same, but ProRes RAW will never replace "normal" ProRes. They are designed for different things.
You will never (except trimming like with R3D) have ProRes RAW exporter on any NLE etc as it's pointless. ProRes RAW is made to hold RAW sensor data with light compression and its place is in cameras or recorders. On NLE etc side you just need a decoder. Only time you would ever want ProRes RAW encoder is for RAW to RAW conversion, e.g. ARRI RAW to ProRes RAW e.g. to save space on RAW assets archive (Cineform had such a tool for converting some RAW formats into CF RAW).
Offline

Andrew Kolakowski

  • Posts: 9206
  • Joined: Tue Sep 11, 2012 10:20 am
  • Location: Poland

Re: ProRes RAW for UMP

PostSat Apr 07, 2018 1:21 pm

Dmitry Shijan wrote:
Michael McCaffrey wrote:
FrankApollonio wrote:REALLY REALLY REALLY WANT THIS ON URSA MINIS .... DROP THAT **** INSTANTLYYYY BMD ... and please drop a new camera at NAB ...and please ship next week. thanks!


YeS! ProRes RAW on the UM4.6K!! I hate cinemadng. It’s so bloated and processor intensive I don’t even shoot raw on my raw camera because it just takes WAY too much data and it won’t even playback on the computer in real-time with a 2013 12-core MacPro and a GTX 1080 Ti eGPU. Can’t stand cinemadng. I’ve been waiting for BMD to release their own codec. Maybe ProRes RAW will fit the bill.


From white paper pdf that ProRes RAW is same size as dng 3:1 and 4:1. It will not helps to save storage space.
Many tech people on forums report that that 3:1 and 4:1 DNG uses same compression method as ProRes.
In white paper pdf there is no any comparison of ProRes RAW with 4:1 and 3:1 compressed DNG. For unknown reason they just ignore that format.
As we know 4:1 and 3:1 compressed DNG introduced by Blackmagic and still not supported in most applications.
I think ProRes RAW may be just some kind of evolution of cinema DNG format for video. I think it may be commonly accepted if allow faster playback and encoding than compressed DNG.


Yep, as at those compression ratios you don't need anything fancy or complex. You want fairly efficient, but simple techniques in order to give good performance. Anything complex (eh RED) will mean decoding nightmare. This is why I prefer Cineform than R3D as possible small RED quality gain is blown by such a huge decoding power needs.
Cinema DNG is a bit of disaster as no one ever really had a real interest in it. It's half baked format.
Offline

Wayne Steven

  • Posts: 3362
  • Joined: Thu Aug 01, 2013 3:58 am
  • Location: Earth

Re: ProRes RAW for UMP

PostSat Apr 07, 2018 2:31 pm

I think Red did a licensing deal for cineform technology and things went low noise and the Redcode revision suddenly worked a lot faster. We also got the lawsuit between Red and Sony seemed to die down, and a deal between Sony and Aptina (where the micro studio's sensor design and from) to licensed each other's sensor technology, and Red and Sony having similar performing sensor tech. The low end threatening craft camera came, and I've read it said Red was involved (somehow I doubt much) and then dissapeared. I doubt, because I don't think Red would leave people out of pocket if they owned all of it, and wouldn't like to if they owned only some if it. Maybe Grant can offer a discount on the 4K pocket to all those craft pre-order buyers who searched waiting for a refund, as a sorry for your loss gesture?
Last edited by Wayne Steven on Fri Apr 13, 2018 1:25 am, edited 1 time in total.
aIf you are not truthfully progressive, maybe you shouldn't say anything
bTruthful side topics in-line with or related to, the discussion accepted
cOften people deceive themselves so much they do not understand, even when the truth is explained to them
Offline

Andrew Kolakowski

  • Posts: 9206
  • Joined: Tue Sep 11, 2012 10:20 am
  • Location: Poland

Re: ProRes RAW for UMP

PostSat Apr 07, 2018 3:39 pm

R3D definitely was (and probably still is) nothing more than JPEG2000.
When ffmpeg introduced decoder for it (which was just a JPEG2000 decoder), RED decided to encrypt files, so only their official SDK can be used. It's all about control (so RED can/could sell you RED Rocket card).

Funny enough that Cineform RAW converter was working on RAW pixels with some formats, but not RED as they did not allow direct access to RAW pixels (even over their SDK as far as I understand). Instead whole debayer process was performed and then RAW data was interpolated and converted to Cineform RAW. Of course whole thing was crazy slow, where other formats were fast.

I don't see anything great in R3D- it's good quality for the price of nightmare decoding. What may be advanced is chip inside RED cameras which allows to compresses data into R3D. Format itself is nothing special and it could be replaced with ProRes RAW (if it had more than 12bit support).
Offline
User avatar

Jamie LeJeune

  • Posts: 2012
  • Joined: Tue Feb 26, 2013 4:33 am
  • Location: San Francisco

Re: ProRes RAW for UMP

PostSat Apr 07, 2018 6:14 pm

John Brawley wrote: I believe it’s been leaked or announced elsewhere that Resolve supports this already. It’s not just FCPx.

Wow, great! That's the first I've heard. I assumed it would come someday down the line, but it's really encouraging to hear that it will be supported in Resolve right from the start.
Apple's white paper did not inspire confidence and made it seem as if ProRes raw would only be readable at release in FCPX and also that FCPX would not give control to make WB and ISO metadata adjustments before debayer.
I assume that Resolve will offer proper control over the metadata at both the project level and the clip level as they do for other raw codecs.

How the metadata is handled through the workflow is an area that I hope Apple and BMD will innovate in as well. Although I'm no fan of Red cameras and .r3d files just bog down a computer, the workflow Red has allowed by leveraging the raw metadata is one area that it would be great if Apple copied. It's great the way with .r3d files, a cinematographer can dial in the desired look in camera using control over the raw metadata (and even more if they add Red CineX to the mix), and what they see there on set, and what the editor sees on screen in the NLE, and what the colorist will see when they open the files to begin the grade, are all automatically the same because the metadata comes through and is read by so many post apps. That's an amazing feat. I haven't used IPP2 much yet, and maybe Red have made it more complicated in the process.
In any case, I really hope Apple ProRes raw will further develop that amount of look control and automatic passing of the intended look seamlessly through post.
www.cinedocs.com
http://www.imdb.com/name/nm4601572/
Offline

Andrew Kolakowski

  • Posts: 9206
  • Joined: Tue Sep 11, 2012 10:20 am
  • Location: Poland

Re: ProRes RAW for UMP

PostSat Apr 07, 2018 7:15 pm

There are so many things which can be done. Simple one- add desired look LUT into actual ProRes RAW file (on recorder/camera level), so with supported software you can later tick e.g. "Final Look" option and this embedded LUT is applied. Metadata (specially active one) is such an underrated/underused thing. Cineform was years ahead with this, but it was ignored.
Post industry is somehow very legacy and so old fashion for me. Sometimes they do advanced things, but then they over-do it, so it's not good either.
Offline
User avatar

rick.lang

  • Posts: 17156
  • Joined: Wed Aug 22, 2012 5:41 pm
  • Location: Victoria BC Canada

Re: ProRes RAW for UMP

PostSun Apr 08, 2018 3:22 pm

Good news if ProRes raw/HQ are supported in Resolve, but that would also raise hopes that’s it’s also supported in the next firmware release for existing cameras. If the only new camera is the BMPCC4K, I expect that camera will offer ProRes raw out of the gate. Followed quickly by firmware updates for the UMP4.6K and then the UM4.6K. Others to follow?

Nice when a new codec innovation is actually easier to manage than an existing codec (assuming there’s room in the firmware storage area).
Rick Lang
Offline

Andrew Kolakowski

  • Posts: 9206
  • Joined: Tue Sep 11, 2012 10:20 am
  • Location: Poland

Re: ProRes RAW for UMP

PostSun Apr 08, 2018 3:38 pm

I'm expecting core technology to be almost the same as in "normal" ProRes, so adding it to existing cameras should not be a problem. If they can do "normal" ProRes then there should be no reason not to do ProRes RAW from a technical point. You save so much processing power on debayering that your only worry is storage speed, which for ProRes is not biggest issue (standard SSDs speeds allow for quite a lot). ProRes RAW won't reduce (at current profiles) storage requirements, but your are getting so much more with most likely boosted quality.

Encoding and recording RAW is way easier than doing it for final video inside camera. It's easier on about every level- processing power, storage, complexity etc.
The only "problem" is a marketing decision for giving RAW data out of "cheaper" cameras. Maybe ProRes RAW will become a trigger which will open this non-existing so far possibility.

It's really all about willingness of giving RAW data to users. You can even do it using h264 (h264 RAW) in DSLRs etc. It's all business decision.
I'm not sure how heavier compression artefacts on RAW assets affect final debayered video, but I assume this is not going to be that much more "visible" than in case of encoding final debayered video. If it's more visible then we can offset it by raising bitrate as we deal only with monochrome 1 channel data, not RGB/YUV one. This allows for substantial quality gain keeping data rate at the same levels as we use now for final video.
Encoding final debayered video in camera feels like a total waste of bits and it's quite intriguing that we have to pay so much to get it. It's all down to fact that camera manufactures don't want to give RAW sensor data as it would allow people to get way better results from low end cameras/DSLRs. Imagine you can get eg. Sony a7S video as RAW. It would mean huge jump in quality!
Offline
User avatar

rick.lang

  • Posts: 17156
  • Joined: Wed Aug 22, 2012 5:41 pm
  • Location: Victoria BC Canada

Re: ProRes RAW for UMP

PostSun Apr 08, 2018 5:05 pm

Andrew Kolakowski wrote:I'm expecting core technology to be almost the same as in "normal" ProRes, so adding it to existing cameras should not be a problem. If they can do "normal" ProRes then there should be no reason not to do ProRes RAW from a technical point. You save so much processing power on debayering that your only worry is storage speed, which for ProRes is not biggest issue (standard SSDs speeds allow for quite a lot). ProRes RAW won't reduce (at current profiles) storage requirements, but your are getting so much more with most likely boosted quality...

The only "problem" is a marketing decision for giving RAW data out of "cheaper" cameras. Maybe ProRes RAW will become a trigger which will open this non-existing so far possibility...

I'm not sure how heavier compression artefacts on RAW assets affect final debayered video, but I assume this is not going to be that much more "visible" than in case of encoding final debayered video...


Generally I agree with your statements with the exception re storage requirements. I’m thinking the camera’s storage media usage will decrease with ProRes raw/HD. You can refer to the original white paper on ProRes and the new white paper on ProRes raw. If you meant storage in post processing, the debayered and rendered storage is likely very similar.

The savings in storage space in the camera is a potentially significant improvement and the savings in processor cycles may lead to support for higher frame rates subject to the recording media.

Your comments on the dominant market players such as Sony and Canon are spot on. They may change, but I’m not holding my breath for that to occur. I’ve given up feeling they’re relevant personally.

Now testing how the ProRes raw/HD performs... can’t wait to update my test video that compared raw and ProRes to include the new codecs.

I hope I’m not disappointed like I was when I compared h.264 and h.265. I was so looking forward to saving space, but I saw that h.265 did not produce quite the quality of h.264 so I only use it when I really need the savings. Your mileage may vary because I’m blind as a bat.




Sent from my iPhone using Tapatalk
Rick Lang
Offline

Aaron Green

  • Posts: 249
  • Joined: Tue Sep 17, 2013 8:08 pm

Re: ProRes RAW for UMP

PostSun Apr 08, 2018 6:06 pm

According to Atomos CEO ProRes RAW is a single video file instead of folders full of still images. Just seems like a cleaner approach than cDNG.
Offline

Nate Porter

  • Posts: 211
  • Joined: Wed May 01, 2013 8:22 am

Re: ProRes RAW for UMP

PostSun Apr 08, 2018 6:37 pm

Aaron Green wrote:According to Atomos CEO ProRes RAW is a single video file instead of folders full of still images. Just seems like a cleaner approach than cDNG.
That's my main issue with cDNG. I have tight turnaround times and it takes too long to manage cDNG compared to prores.
Offline

Andrew Kolakowski

  • Posts: 9206
  • Joined: Tue Sep 11, 2012 10:20 am
  • Location: Poland

Re: ProRes RAW for UMP

PostSun Apr 08, 2018 6:41 pm

rick.lang wrote:
Andrew Kolakowski wrote:The savings in storage space in the camera is a potentially significant improvement and the savings in processor cycles may lead to support for higher frame rates subject to the recording media.

Sent from my iPhone using Tapatalk


Bandwidth savings depend on what you compare it to. Average rate for ProRes RAW HQ is 15% above ProRes HQ (and about the same situation for normal mode). No saving here.
ProRes RAW HQ should save about 20% bandwidth compared to ProRes444 though.


I would not count on compression efficiency boost (but quality can increase thanks to monochrome nature of RAW data) as ProRes RAW will be most likely based on the "old" engine.
Some changes were made in terms of sensitivity and average/min/max bitrate values, but core math is probably the same. Without Apple comment it's hard to say for sure, but even fact that only 12bit is supported atm. is some indication about old engine.
In the same time I don't think that much more can/should be done- if you start making it more complex you will lose on decoding speed. The only valid refinement could be possible thanks to fact that bayer data is monochrome (and maybe this was done).

There is a possibility (in my opinion very small) that a whole new engine was developed, but I would still not count for massive quality difference compared to e.g. R3D or Cineform. One of the reasons for this are quoted decoding speeds, which indicate that algorithm is fairly simple. Fact that Atoms and other manufactures are going to provide it "so quickly" also suggest old engine, which means that companies can fairly quickly adjust their existing hardware encoders for ProRes RAW mode. White paper is also vague about it and I think if it would be new and shiny algorithm Apple would say so and maybe even give it new name.

For me it looks like just a quick job of adding RAW mode to old and proven ProRes engine. There is no information atm. which would suggest otherwise. If anything surprises me is fact that it has not be done earlier. CinemaDNG is just a half baked product and there was always massive need for some RAW video data standard.

For those who says it's proprietary- well R3D is 10x worse as it's also encrypted. Future will tell, but I expect many free tools (aside of Apple SDK) for decoding ProRes RAW. I don't see fact it's Apple developed being real issue here as we just need decoder on NLE side. It's quite a different than normal ProRes and mainly affects companies which produce cameras/recorders as it's them who will have to fight for licensing. I don't really think Apple will be making any problems with this and in the same time I hope they will be strict about implementations, so we will have good reliability.
Next

Return to Cinematography

Who is online

Users browsing this forum: Alex Mitchell and 29 guests