Why are you bothering setting up something to key? Just make it with an alpha and save as OpenEXR, and it's done—no keying necessary. Or if you need it to be in a video format, there are flavors of both ProRes and DNxHR that support alpha.
If you have a raster image, enlarging it will always reduce the quality. Instead, make your element at the largest anticipated size and expect to shrink it. There will still be some filtering, but it's way easier to remove data than to invent it, so you will maintain sharpness more easily. You can choose the Filter Method in a Transform tool to choose one that works better for your particular element. Linear will frequently soften things the most, but it's the fastest. If you go with Catmull-Rom or higher, you'll get a tiny bit of artificial sharpening, which might cause ringing if you have a lot of contrast or extremely saturated colors. Most of the time, though, Catmull-Rom is the one I tend to pick.
Are you zooming in with the Viewer? Or using a Transform node to increase the size? The viewer, by default, has a feature called "Smooth Resize" turned on, which attempts to interpolate pixels when you zoom in instead of showing you an accurate representation. I forget exactly where it is in the context menu, but if you right-click in a Viewer and explore around in there, you should be able to find where to turn it off. But remember that if you're not viewing at 100%, you're not seeing what the actual output will be.