I used the format "Burn into video" when rendering. It shouldn't be possible for the subtitles to have been burnt into the video in the timeline as well, but if I disable the subtitle track I see the original ones without any shadows or duplicates.
I did get an option to name the item in the render queue differently since there was an existing one with the same name. To my knowledge that name points to the name of a previously rendered movie and has nothing to do with what is in the timeline.