If you can, share a closeup screenshot of the artifacts you're seeing at the edge of the frame. I suspect it's the same thing I ran into when I started using Fusion for match moving:
viewtopic.php?f=22&t=66289
As far as I can tell there is just some bad math in the antialiasing calculations Fusion does. Which is ironic because Resolve also has terrible antialiasing, but on a completely different code base...
I'd be happy if someone could inform me that I'm wrong and this is a workflow error, but that wasn't the impression I got when I first asked around here and on WSL.
The only real solution I came up with was to dilate around the edges with a tiny 1 or 2 pixel mask to sort of "fake" the clipping on the planar transform.