Fusion 8.2 with Deadline 7.2 & Deadline 8.x

Hi there,

Anyone got Fusion 8.2 working with Deadline yet? We're currently giving Fusion Studio 8.2 a bit of a test on our farm here but there seems to be an error that we see within the Deadline Slave Client that causes the job to fail to start.

The error is :

Code: Select all

2016-10-21 14:49:20:  0: INFO: Process exit code: -1073741819


Here's the log:

Code: Select all

2016-10-21 14:49:18:  0: Start Job timeout is disabled.
2016-10-21 14:49:18:  0: Task timeout is 1170 seconds (Regular Task Timeout)
2016-10-21 14:49:18:  0: Loaded job: [A9017] : Simple_Test : Fusion comp job (580a1a9080ef932220478ed3)
2016-10-21 14:49:18:  0: INFO: Executing plugin script C:\Users\renderman\AppData\Local\Thinkbox\Deadline7\slave\quad-42\plugins\580a1a9080ef932220478ed3\FusionCmd.py
2016-10-21 14:49:18:  0: INFO: About: Fusion Cmd Plugin for Deadline
2016-10-21 14:49:18:  0: INFO: The job's environment will be merged with the current environment before rendering
2016-10-21 14:49:18:  0: Plugin rendering frame(s): 1-5
2016-10-21 14:49:18:  0: PYTHON: **VERSION:**
2016-10-21 14:49:18:  0: INFO: Stdout Redirection Enabled: True
2016-10-21 14:49:18:  0: INFO: Stdout Handling Enabled: True
2016-10-21 14:49:18:  0: INFO: Popup Handling Enabled: False
2016-10-21 14:49:18:  0: INFO: Using Process Tree: True
2016-10-21 14:49:18:  0: INFO: Hiding DOS Window: False
2016-10-21 14:49:18:  0: INFO: Creating New Console: False
2016-10-21 14:49:18:  0: INFO: Running as user: renderman
2016-10-21 14:49:18:  0: INFO: Executable: "C:/Program Files/Blackmagic Design/Fusion Render Node 8/FusionConsoleNode.exe"
2016-10-21 14:49:18:  0: INFO: Argument: "P:\A9016_Axis_Pipeline2016\user\Jamie\Fusion\8.1_tests\Simple_Test.comp" /render /start 1 /end 5 /step 1 /verbose /quiet /quietlicense /clean /quit /log
2016-10-21 14:49:18:  0: INFO: Startup Directory: "C:\Program Files\Blackmagic Design\Fusion Render Node 8"
2016-10-21 14:49:18:  0: INFO: Process Priority: BelowNormal
2016-10-21 14:49:18:  0: INFO: Process Affinity: default
2016-10-21 14:49:18:  0: INFO: Process is now running
2016-10-21 14:49:18:  0: STDOUT: ---------------------------------------------------
2016-10-21 14:49:18:  0: STDOUT: Starting Fusion Render Node 8.2 at 21/Oct/16 14:49:18
2016-10-21 14:49:18:  0: STDOUT: C:/Program Files/Blackmagic Design/Fusion Render Node 8\FusionConsoleNode.exe
2016-10-21 14:49:18:  0: STDOUT: ---------------------------------------------------
2016-10-21 14:49:18:  0: STDOUT: Initialising QuickTime
2016-10-21 14:49:18:  0: STDOUT: Checking for licenses...
2016-10-21 14:49:18:  0: STDOUT: Loading support DLLs
2016-10-21 14:49:18:  0: STDOUT: Loading fusionoperators.dll
2016-10-21 14:49:18:  0: STDOUT: Loading fusionformats.dll
2016-10-21 14:49:18:  0: STDOUT: Loading Plugins
2016-10-21 14:49:18:  0: STDOUT: Loading plugin C:/Program Files/Blackmagic Design/Fusion Render Node 8/Plugins/Blackmagic/3D.plugin
2016-10-21 14:49:18:  0: STDOUT: Loading plugin C:/Program Files/Blackmagic Design/Fusion Render Node 8/Plugins/Blackmagic/Alembic.plugin
2016-10-21 14:49:18:  0: STDOUT: Loading plugin C:/Program Files/Blackmagic Design/Fusion Render Node 8/Plugins/Blackmagic/CinemaRaw/CinemaRaw.plugin
2016-10-21 14:49:18:  0: STDOUT: Loading plugin C:/Program Files/Blackmagic Design/Fusion Render Node 8/Plugins/Blackmagic/DDS.plugin
2016-10-21 14:49:18:  0: STDOUT: Loading plugin C:/Program Files/Blackmagic Design/Fusion Render Node 8/Plugins/Blackmagic/Dimension.plugin
2016-10-21 14:49:18:  0: STDOUT: Loading plugin C:/Program Files/Blackmagic Design/Fusion Render Node 8/Plugins/Blackmagic/DirectShow.plugin
2016-10-21 14:49:18:  0: STDOUT: Loading plugin C:/Program Files/Blackmagic Design/Fusion Render Node 8/Plugins/Blackmagic/FBX.plugin
2016-10-21 14:49:18:  0: STDOUT: Loading plugin C:/Program Files/Blackmagic Design/Fusion Render Node 8/Plugins/Blackmagic/Fuses.plugin
2016-10-21 14:49:18:  0: STDOUT: Loading plugin C:/Program Files/Blackmagic Design/Fusion Render Node 8/Plugins/Blackmagic/OMF.plugin
2016-10-21 14:49:18:  0: STDOUT: Loading plugin C:/Program Files/Blackmagic Design/Fusion Render Node 8/Plugins/Blackmagic/OpenColorIO.plugin
2016-10-21 14:49:18:  0: STDOUT: Loading plugin C:/Program Files/Blackmagic Design/Fusion Render Node 8/Plugins/Blackmagic/OpenEXR.plugin
2016-10-21 14:49:18:  0: STDOUT: Loading plugin C:/Program Files/Blackmagic Design/Fusion Render Node 8/Plugins/Blackmagic/OpenFX.plugin
2016-10-21 14:49:18:  0: STDOUT: Loading plugin C:/Program Files/Blackmagic Design/Fusion Render Node 8/Plugins/Blackmagic/Paint.plugin
2016-10-21 14:49:18:  0: STDOUT: Loading plugin C:/Program Files/Blackmagic Design/Fusion Render Node 8/Plugins/Blackmagic/Particles.plugin
2016-10-21 14:49:18:  0: STDOUT: Loading plugin C:/Program Files/Blackmagic Design/Fusion Render Node 8/Plugins/Blackmagic/QuickTime.plugin
2016-10-21 14:49:18:  0: STDOUT: Loading plugin C:/Program Files/Blackmagic Design/Fusion Render Node 8/Plugins/Blackmagic/Text.plugin
2016-10-21 14:49:18:  0: STDOUT: Loading plugin C:/Program Files/Blackmagic Design/Fusion Render Node 8/Plugins/Blackmagic/Utilities.plugin
2016-10-21 14:49:18:  0: STDOUT: Initialising Class Registry
2016-10-21 14:49:20:  0: INFO: Process exit code: -1073741819
2016-10-21 14:49:21:  0: An exception occurred: Error: Renderer returned non-zero error code -1073741819. Check the console slave's output.
2016-10-21 14:49:21:     at Deadline.Plugins.ScriptPlugin.RenderTasks(String taskId, Int32 startFrame, Int32 endFrame, String& outMessage, AbortLevel& abortLevel) (Deadline.Plugins.RenderPluginException)

Quick update. We're seeing similar behaviour in Deadline 8.x. We've raised this with Thinkbox and also informed BMD support.

Would be good to hear how FusionConsoleNode.exe plays with parent launching thread on a Windows machine level. Starting to worry that this won't be an easy one to get resolved.

cheers

Jamie.

Hey Jamie,

Are your frames rendering before the error code?

It's 32 days and counting since I posed the question to BM 'what this error code refers to'. My local agent has followed up with BM multiple times, but I have never heard so much as a peep back.

Some have found success with previous beta builds, but updates/downgrades that have fixed this problem for some, have caused this error for others.

For me, the frames render before the exit code. Rather than telling my render manager (Muster) to ignore errors, I inserted 1073741819 as an expected exit code (ADD DEFAULT_RETCODE -1073741819). Then it just accepts that code as being normal. My render manager supplier tells me the code is Fusion generated and is not a problem with the manager itself.

Render nodes have a .prefs file, and I have spent hours editing that to try and resolve the error, but no luck. I have a funny feeling it may be related to OpenCL being called on nodes that don't have an OpenCL device - but that hunch is based on nothing more than the same error code being used for Maya OpenCL errors.

Hey Sam, do you get the error after each frame renders or after each set of frames?

I'm just wondering, because we use Royal Render, and have errors that come up but the frames render fine. It seems to be when a client finishes its frame range, and quit outs before starting back up a a new frame range.
Like you, I believe it has to do with gpu. To me it's almost like the gpu memory gets stuck and it has to crash out before it can start back up or something.

Hey, The error is returned after the packet of frames (typically 4) has finished rendering. I'm not sure if fusion typically logs errors to the manager for each frame or is just providing a summary as the chunk completes. If it is an error on closing down the current packet that would be interesting.

To be more specific, this is the error code we are getting:

Code: Select all

R 12| ++++ Executable returned -1073741819 (0x c0000005) as exit code for frame 1009.
R 13| Error: Renderer or plugin contains a bug. Crash was caused by a NULL pointer memory access.

Minus the frame number of course.

I think we first started getting these errors with Fusion 7.7.1. We reported it to BMD at the time, but it it didn't get much traction, as they thought it was coming from the render manager ( Royal Render). Now that others are seeing this with their managers, it might get looked at.

Hi all,

Windows 10 update broke Fusion 6.4 so we finally bought Fusion 8... and cannot get it to work with Deadline at all.

Same deal:

Using Fusion 8.2 build 2 and the latest Deadline 8.0.11.2

Code: Select all

=======================================================
Error
=======================================================
Error: Renderer returned non-zero error code -1073741819. Check the console slave's output.
   at Deadline.Plugins.PluginWrapper.RenderTasks(String taskId, Int32 startFrame, Int32 endFrame, String& outMessage, AbortLevel& abortLevel)

=======================================================
Type
=======================================================
RenderPluginException

=======================================================
Stack Trace
=======================================================
   at Deadline.Plugins.SandboxedPlugin.a(DeadlineMessage A_0)
   at Deadline.Plugins.SandboxedPlugin.RenderTask(String taskId, Int32 startFrame, Int32 endFrame)
   at Deadline.Slaves.SlaveRenderThread.a(TaskLogWriter A_0)

=======================================================
Log
=======================================================
2016-11-10 14:57:58:  BEGIN - X99A\dave
2016-11-10 14:57:58:  0: Executing plugin command of type 'Initialize Plugin'
2016-11-10 14:57:58:  0: INFO: Executing plugin script 'C:\Users\dave\AppData\Local\Thinkbox\Deadline8\slave\x99a\plugins\5823ef0225689a53f0ef2e3a\FusionCmd.py'
2016-11-10 14:57:58:  0: INFO: About: Fusion Cmd Plugin for Deadline
2016-11-10 14:57:58:  0: INFO: The job's environment will be merged with the current environment before rendering
2016-11-10 14:57:58:  0: Done executing plugin command of type 'Initialize Plugin'
2016-11-10 14:57:58:  0: Start Job timeout is disabled.
2016-11-10 14:57:58:  0: Task timeout is disabled.
2016-11-10 14:57:58:  0: Loaded job: Network render test cmd (5823ef0225689a53f0ef2e3a)
2016-11-10 14:57:58:  0: Executing plugin command of type 'Start Job'
2016-11-10 14:57:58:  0: Done executing plugin command of type 'Start Job'
2016-11-10 14:57:58:  0: Plugin rendering frame(s): 0
2016-11-10 14:57:59:  0: Executing plugin command of type 'Render Task'
2016-11-10 14:57:59:  0: INFO: Stdout Redirection Enabled: True
2016-11-10 14:57:59:  0: INFO: Stdout Handling Enabled: True
2016-11-10 14:57:59:  0: INFO: Popup Handling Enabled: False
2016-11-10 14:57:59:  0: INFO: Using Process Tree: True
2016-11-10 14:57:59:  0: INFO: Hiding DOS Window: False
2016-11-10 14:57:59:  0: INFO: Creating New Console: False
2016-11-10 14:57:59:  0: INFO: Running as user: dave
2016-11-10 14:57:59:  0: INFO: Executable: "C:\Program Files\Blackmagic Design\Fusion Render Node 8\FusionConsoleNode.exe"
2016-11-10 14:57:59:  0: INFO: Argument: "S:\FLOWS\Network render test.comp" /render /start 0 /end 0 /step 1 /verbose /quiet /quietlicense /clean /quit
2016-11-10 14:57:59:  0: INFO: Full Command: "C:\Program Files\Blackmagic Design\Fusion Render Node 8\FusionConsoleNode.exe" "S:\FLOWS\Network render test.comp" /render /start 0 /end 0 /step 1 /verbose /quiet /quietlicense /clean /quit
2016-11-10 14:57:59:  0: INFO: Startup Directory: "C:\Program Files\Blackmagic Design\Fusion Render Node 8"
2016-11-10 14:57:59:  0: INFO: Process Priority: BelowNormal
2016-11-10 14:57:59:  0: INFO: Process Affinity: default
2016-11-10 14:57:59:  0: INFO: Process is now running
2016-11-10 14:58:04:  0: INFO: Process exit code: -1073741819
2016-11-10 14:58:04:  0: Done executing plugin command of type 'Render Task'

=======================================================
Details
=======================================================
Date: 11/10/2016 14:58:06
Frames: 0
Elapsed Time: 00:00:00:10
Job Submit Date: 11/10/2016 14:52:34
Job User: dave
Average RAM Usage: 13758324736 (21%)
Peak RAM Usage: 13758324736 (21%)
Average CPU Usage: 14%
Peak CPU Usage: 63%
Used CPU Clocks (x10^6 cycles): 39808
Total CPU Clocks (x10^6 cycles): 284337

=======================================================
Slave Information
=======================================================
Slave Name: X99A
Version: v8.0.11.2 Release (ed3c7aaea)
Operating System: Windows 10 Pro
Running As Service: No
Machine User: dave
IP Address: 192.168.0.236
MAC Address: 08:62:66:7C:EC:DB
CPU Architecture: x64
CPUs: 16
CPU Usage: 41%
Memory Usage: 12.8 GB / 63.9 GB (20%)
Free Disk Space: 857.947 GB (19.508 GB on C:\, 27.504 GB on D:\, 757.155 GB on P:\, 53.780 GB on S:\)
Video Card: NVIDIA GeForce GTX 770


For me, the frames render before the exit code. Rather than telling my render manager (Muster) to ignore errors, I inserted 1073741819 as an expected exit code (ADD DEFAULT_RETCODE -1073741819). Then it just accepts that code as being normal.

Hey Sam thanks for the inspiration - turns out Fusion is rendering the frames, just reporting them as failed. So if I set the job error limit to something higher than the number of frames we are good...

Hacky, but finally working!

Anyone got advice for a dummy on how to add this error code as benign in Deadline, so I don't have to edit each job?

EDIT: Solved!
Found a block of code here that works with one tweak. (delete undefined "LogInfo") http://docs.thinkboxsoftware.com/products/deadline/5.2/User%20Manual/manual/aftereffects.html

If I add this code to my FusionCmd.py in [Repository]/plugins/FusionCmd it works...

Code: Select all

    def CheckExitCode( self, exitCode ):
      if exitCode != 0:
         if exitCode == -1073741819:
            ( "Ignoring exit code -1073741819" )
         else:
            FailRender( "Renderer returned non-zero error code %d." % exitCode )


Entire working FusionCmd.py downloadable here https://dl.dropboxusercontent.com/u/10143749/FusionCmd.py

Should add disclaimer - this is my very first Python script edit ever, so use at own risk, and do not blame moi should the apocalypse ensue.

Hi David,

That's good you found a way to bypass only that code. Here's hoping someone can get some direction from BM. If you ever find a way to get any support, please let me know.

Hi Sam,

Are your frames rendering before the error code?

Yeap, they seem to be indeed completing before this exit code presents itself.

I do agree that there's something possibly module wise that's being loaded but not handled correctly, such as OpenCL checks, causing this.

We've previously also gave Thinkbox feedback on allowing the exception in their plugin as a workaround.

We're testing with this temp fix in place but would be nice to see it not exit code in this way at all if possible BMD

Cheers!