Sorry to say, but this one is pretty annoying and took me a couple hours to figure out...:-/
It goes like this:
- Open the comp
- View the BG, activate the PolyMask (so that you can see it's outline)
- Press play and see the actual Mask fit the Polyline (woohoo!)
- Now press "Render" and see how the Mask becomes *completely* useless
although the "outline" still seems to move correctly....

Workaround:
Take a BG and attach it to a CornerPositioner,
animating the Corners the same way as the polygon.
Cheesy, but works in this case.
Please fix this at your earliest convenience.
Thanks.