Can Resolve 'mask' subtitles?

Hello.

I have a video that has embedded subtitles at some points (it isn't possible to get a copy without them at this point).

They appear mostly in the mattes of the video. I've been able to cover those up with a colour mask. However, they sometimes go above the mattes and into the picture.

I don't necessarily want to crop the picture, so is there a way I could 'mask' the subs but without covering any of the image?

Thank you!

I will start by saying that this is definitely not a particularly fun problem to have to solve, but I do have a couple ideas that might help. First, if you can key/qualify the subtitles (shouldn't be too hard, since they're probably solid white/black), you can remove them from the image so that you're left with alpha where they used to be. That way, you can work on a copy of the footage underneath it and simply find a way to fill the alpha space where the subtitles used to be.

For this, the two tools I would reach for first are the Paint node's clone multistroke tool, and the TimeSpeed node, probably in combination with Transform. I actually haven't used clone multistroke for this before, but TimeSpeed has recently become a very good friend of mine for getting rid of unwanted objects. If you're lucky enough to have a part of the same shot without the subtitles in it, simply offset the underlying footage to that point and put it beneath the cut out subs. If it's a stationary shot, you can still do this even if you only have one clean frame of the area you need to replace; just set the speed to zero and add some noise so it matches the foreground footage.

If you don't have any footage of that shot without the subtitles, but the camera moves, you can use TimeSpeed to offset it to a clean section, then use a Transform node to move it back under the correct part of the image. In less ideal cases (e.g. tripod panning) you might also be able to run the offset image through an ImagePlane3D and Render3D node so you can match the perspective. (DVE also has similar controls and has the performance benefit of not having to be run through the 3D engine, but in my experience ImagePlane3D gives better results more consistently.)

Also, it might be a good idea to find a particularly difficult clip to work on first, i.e. something with a lot of texture/variation behind the subs, so that you don't get a lot of work done on other clips first only to realize later that you can't fix everything you need to fix.

Other less than ideal possibilities include fixing the easy clips, and then fixing the harder clips as best you can, and then cropping in on only those clips. You might even be able to get away with a mediocre removal of the subs in the difficult clips, cropping in partially, and then blurring the remnants of the subs you had trouble painting out fully.

Hope some of this is at least partially helpful. And if it is, I'd be curious to know what ended up working for you. Good luck!

Thank you for such a detailed reply! There's a lot to take in but I look forward to digging into it and getting somewhere.

In view of the informative reply from TCP786, is it possible to rotoscope part of the picture from one source to another to cover the subtitles?

I think that's a bit different from what's mentioned, but would it also be a viable option? I'm getting to grips with what to do and it's a fun process.

it's a fun process

Haha. Sorry, I'm just cracking up because a couple days ago I was rotoscoping and complaining to a couple friends about how not fun it is. But in seriousness, I consider roto to be a process that basically exists for situations where you need to make a mask/matte with a complex enough edge/shape (and often one that changes every frame) that it can only realistically be done by hand. In the case of removing subtitles, since they don't move or change color at all, to me it's more of an "at least I don't have to rotoscope this!" situation. In your case, hopefully you can get a useful key with just a few clicks and keypresses.

However, it's still likely that the edges of the removed subtitles could be rough, and rotoscoping them is a great solution to that. In this case, I would still make a background layer (with clone paint or timeSpeed), and then use a new paint node to create a layer mask for the merge node that puts it under the original with the cut out subtitles. Then I'd use that paint node to add to the edges of the cutout area so that the transition from the foreground to the background is smooth, and this very well might be something you need to do on a frame by frame basis.

Yikes, and to think we got all the way here from something that started as "If I had the right files, I'd just untick a checkbox." Regardless, I wish you the best and most prompt success.

Also this is just the first thing that I thought of as a solution to this, so if anyone else has any other methods to consider, or reasons not to use the ones I suggested, please chime in. (Although for what it's worth, I did use this technique on a few clips last week and I'm happy with the results.)