Jump to: Board index » General » Fusion

Very slow EXR file caching

Learn about 3D compositing, animation, broadcast design and VFX workflows.
  • Author
  • Message
Offline

Ludvik Koutny

  • Posts: 96
  • Joined: Mon Nov 10, 2014 10:52 am

Very slow EXR file caching

PostMon Dec 15, 2014 11:18 am

Hi,

i've got EXR sequence of 2560*1080 32bit full float images, stored as tiles with ZIPS compression. Fusion seems to load and cache them terribly slow. It takes about 8 seconds per frame to cache, while PDplayer takes about 1 second to cache them from the same location.

Now, when i enable Local cache in import panel, caching speed becomes a lot better, on par with PDplayer (about 1s/frame), but it starts to eat up my SSD drive space rapidly. I am a bit confused this, because caching sequence to RAM memory should be a lot faster than caching it to the hard drive.

What am i doing wrong?

Thanks...
https://www.artstation.com/rawalanche
Offline
User avatar

Marek Subocz

  • Posts: 65
  • Joined: Mon Nov 10, 2014 9:21 pm

Re: Very slow EXR file caching

PostMon Dec 15, 2014 4:34 pm

on a side note - Comping World works smarter with scanline EXR. when using ROI, Fusion would be able to get only the specified region from the file instead of the whole file. The drawback of scanline is the playback issue. AFAIK the best compression for fast exr playback is b44/b44a - but those are tiles, not scanlines.
Offline
User avatar

Michael Wolf

  • Posts: 42
  • Joined: Tue Nov 11, 2014 10:32 am
  • Location: Stuttgart, Germany

Re: Very slow EXR file caching

PostTue Dec 16, 2014 10:04 am

Marek Subocz wrote:on a side note - Comping World works smarter with scanline EXR.

ZIPS compresses one scanline at a time, which is ideal for Fusion, especially when using proxy mode (which can thus extract single scanlines).
ZIP compresses 16 scanlines together as a block - superior compression, better speed - but only if the complete image is loaded at once.
Marek Subocz wrote:AFAIK the best compression for fast exr playback is b44/b44a - but those are tiles, not scanlines.

They can be either. The compression mode of EXRs is independent of the storage layout of the pixel data as tiles or scanlines. EXR can mix and match them freely.

Cheers,
Mike
Michael Wolf - db&w GbR
http://www.db-w.com
Offline

Ludvik Koutny

  • Posts: 96
  • Joined: Mon Nov 10, 2014 10:52 am

Re: Very slow EXR file caching

PostTue Dec 16, 2014 10:08 am

So no one has an answer for me? I just can not understand why caching to local drive is so much faster than caching directly to RAM. It should be opposite behavior. :?
https://www.artstation.com/rawalanche
Offline
User avatar

michael vorberg

  • Posts: 887
  • Joined: Wed Nov 12, 2014 8:47 pm
  • Location: stuttgart, germany

Re: Very slow EXR file caching

PostTue Dec 16, 2014 10:16 am

local caching means that fusion is copying the files to the local cache drive and reads it from there into the ram. this should be a little slower on the first run (because it has to copy the files) but after that it will read it from the (hopefully) faster local disk cache instead of a slow network drive or other slower drives
Offline
User avatar

Michael Wolf

  • Posts: 42
  • Joined: Tue Nov 11, 2014 10:32 am
  • Location: Stuttgart, Germany

Re: Very slow EXR file caching

PostTue Dec 16, 2014 11:25 am

@Ludvik - sorry, I don't. What kind of EXRs are you testing this with (Number of channels, compression mode)?

Cheers,
Mike
Michael Wolf - db&w GbR
http://www.db-w.com
Offline

Jiri Sindelar

  • Posts: 28
  • Joined: Mon Nov 10, 2014 9:53 pm

Re: Very slow EXR file caching

PostTue Dec 16, 2014 12:17 pm

@Ludvik - maybe if you can upload one frame somewhere, I can test the read speed here. Dpx files are faster in Fusion then Exr, but not "that" slow.
Offline

Ludvik Koutny

  • Posts: 96
  • Joined: Mon Nov 10, 2014 10:52 am

Re: Very slow EXR file caching

PostThu Dec 18, 2014 6:02 pm

Hi, sorry for the delay. I've described EXR parameters i use in my original post (2560*1080 32bit full float images, stored as tiles with ZIPS compression).

Unfortunately i can not share the sequence as it's VFX for movie that's still under NDA, but i can try to make some synthetic sequence with same parameters.
https://www.artstation.com/rawalanche
Offline

Jiri Sindelar

  • Posts: 28
  • Joined: Mon Nov 10, 2014 9:53 pm

Re: Very slow EXR file caching

PostFri Dec 19, 2014 11:25 am

Hi,
here are few not very scientific speed tests:

About 100 frames of 32bit float exr files 2560x1080, zips compression, rgb, about 30MB per frame, created in Nuke. Same files were on local SSD and network location (medium traffic, 1Gbps).
Windows 7 Ultimate sp1, Xeon E5-1620@3.7GHz, 64 GB RAM, Nvidia GTX 780 (335.23 driver).


In Fusion comp was set to 32bit float, one viewer on 50% zoom, gamut lut set to sRGB. Local caching turned off, clearing cache after every test.

In Nuke, one sRGB viewer on 50% zoom, full frame processing on (so Nuke reads whole image from drive). ocal caching turned off, clearing cache after every test.


Fusion 6.4 local drive: 6 FPS
Fusion 6.4 network drive: 2.6 FPS
(Tweaks preference: no i/o cancelling, no direct reads, 4 read-ahead buffers)

Fusion 7.5 free local drive: 6.4 FPS
Fusion 7.5 free network drive: 2.6 FPS

Nuke 8.0v6 local drive: 1.6 FPS
Nuke 8.0v6 network drive: 0.4 FPS

Nuke 9.0v1 local drive: 8 FPS
Nuke 9.0v1 network drive: 1 FPS

Hope it helps

Jirka

Return to Fusion

Who is online

Users browsing this forum: No registered users and 5 guests