Desperate need for "Fault Tolerance" in Render Queue items

I set up a transcode of 14 hours worth of footage last night, only to wake up this morning to an error popup in the Render Queue, informing me that Resolve was unable to decode a file. The error apparently popped up only about 1 hour into the rendering process, after I had gone to bed.



The worst thing about this error is that it's a modal warning dialog that actually STOPS the entire Render Queue process, and doesn't try to continue on until a human being sees the error and manually dismisses the error popup, whereupon the Render Queue continues rendering other jobs in the queue. However, it will NOT resume rendering the actual Queue item that contains the errant file!

This is usually not a big deal when rendering out a :30 commercial. But when you are counting on Resolve to transcode hours and hours of footage in an unattended, overnight Render Queue, this is downright catastrophic, especially if you have tight delivery turnaround times on a transcode job. Needless to say, I was not a happy camper when I was greeted by the error this morning.

Resolve really needs to have a "fault tolerance" mechanism for cases like this. If it encounters a decode or write error on a clip in a Queue, it should still continue processing all the other clips in the queue with the assumption that the error was an isolated error. Then ONLY at the end of the rendering process, should it warn you that errors were encountered, and ideally it would also write out an actual error log file listing the specifics of each error, so the user can use the logfile to diagnose each problem clip separately. This way, even if a batch has errors in it, at least it would still successful churn through all the other clips which don't have any issues, and would prevent huge chunks of lost time.

This is also tangentially related to Resolve's poor error-reporting/logging in other areas of the app, specifically the Media Management panel.

+1

It would be great if it had a pop up window at the end with all the unfinished renders and thumbnails of the culprit frames.

+1

It would be great if it had a pop up window at the end with all the unfinished renders and thumbnails of the culprit frames.

I would find it more useful to have an actual, machine-parseable text file log (so you could write scripts that process the error log), with the exact timecode or framecount offset where the error was detected, if possible. I'm not really sure how useful thumbnails would be, because if they can't decode properly, what would there be to show as a thumbnail?

I would guess, as you see them in your timeline. Could be a frame or the clip thumbnail.

What I mean, it could be useful to have a visual idea of the clip that is giving the problem and not just the coordinates pointing to it.

I have no clue how to write scripts and not very proficient at converting frame count to actual reels timecode.

Also it would be cool if we could resume stoped renders. Let's say you paused or your computer crashed when at 97%. To be able to take it from there, instead of starting from scratch... that's what I like about exporting frame sequences.

Also it would be cool if we could resume stoped renders. Let's say you paused or your computer crashed when at 97%. To be able to take it from there, instead of starting from scratch... that's what I like about exporting frame sequences.

This I completely agree with. After all, Resolve is basically a glorified, highly specific GUI over an SQL based database. One would think that it should be able to know what it has rendered in the past (even across different projects), what renders have failed etc, and give you options as to how to proceed with a Render Queue that may conflict with what already exists at a specific folder on the filesystem.

+1

It's a big DaVinci benefit that it stops rendering a corrupted item. But it could skip the corrupted item and render other items in the queue. An error pop-up window may not appear until the entire queue has rendered, and the icon and processing information may appear in the nodes on the right.

I don't like this idea. If there's an error, I want to know right away.

I don't like this idea. If there's an error, I want to know right away.

So you prefer not rendering to rendering? There can be a checkbox for users who want to babysit resolve every step if the way.

If there's an error, then yes I prefer to stop the render and handle it.

If there's an error, then yes I prefer to stop the render and handle it.

So if you have 14 hours of rushes to transcode, and only 1 of the clips out of 2000 clips fails to decode at 1:00 AM while you’re sleeping, and you have to deliver those proxies to post-production first thing in the morning, you’d rather that the rest of the clips NOT get rendered because only 1 clip out of the entire set of data may be corrupted?

Sorry, but that makes absolutely no sense.

It's a big DaVinci benefit that it stops rendering a corrupted item. But it could skip the corrupted item and render other items in the queue. An error pop-up window may not appear until the entire queue has rendered, and the icon and processing information may appear in the nodes on the right.

I totally get where Jim is coming from, as far as wanting to know when errors have occurred as they happen. I am not advocating for the contrary.

I do think Resolve should still warn you in real-time when an error has occurred, but just not in a modal dialog that blocks the rest of the queue from rendering. I can’t think of a good use case for that behavior, perhaps with the possible exception of a situation where the queue may overwrite files that already exist (in which case it already warns you when you click the Render button), or if you may clobber source files (which Resolve should be hard coded NOT to do under any circumstance)





Sent from my iPhone using Tapatalk Pro

I do think Resolve should still warn you in real-time when an error has occurred, but just not in a modal dialog that blocks the rest of the queue from rendering.

I have no problem with real-time alerts. If the processing is stopped, a notification will appear immediately in the node and DaVinci will start processing the next node in the queue. The pop-up at the end of the queue notifies you once again that one or more nodes have stopped processing during rendering. But you will have an error notification in the node immediately.

The way DaVinci warns us is not that important. It is important that one rendering node does not stop the entire queue.

Alerts are important, real time or not.

However, an alert should NOT block the rest of the render queue. Like the OP, I had queue with 150 jobs to output. I set it up, started it up and went out to do other things. When I returned, the whole process had stopped because of one error in one queued job. I would have much preferred that it continued and did all of the jobs in the queue until it finished, and jumped over any with errors (yes, pop up a dialog panel or write a log file flagging the jobs that failed so they can be re-run).

Had this been an overnight job, I would have missed the deadline.

Computers excel at automating repetitive tasks, to free us up to focus on other things. I don't want to have to babysit Resolve when I've set up a batch render queue and go off to take care of other things while it's working.

However, an alert should NOT block the rest of the render queue

As an addendum to this, this morning, I experienced the same issue on yet another overnight render. Except this time, there were no actual errors in the read/write process of the Render Queue. In this case, the issue was a spurious "Remote Grading" error popup dialog (which I've been getting randomly over the past month for some reason, as I reported in this thread), which popped up overnight.

The popup caused the Render Queue to pause. Dismissing the notification caused the Queue to continue rendering.

This is clearly undesirable default behavior for an active Render Queue, and really should be fixed.

+1 for having a machine-parseable text file log with real time alerts if an error is encountered during rendering (maybe with a sound alert?). Also, no disturbing pop-ups during rendering, and always trying to continue the rendering process.

Alerts are important, real time or not.

However, an alert should NOT block the rest of the render queue.

Yeah, agreed, the simple answer is just to have the alert pop up, visually, while the render queue moves on. For single-item exports, like a final delivery, this is fundamentally the same. Error pops up, queue checks for next item, there are no more, queue stops.

But on a batch process I would really, really, really rather just deal with the stuff that failed rather than stop everything.