Audio Levels Question.. HDMI/SDI adapters

Running into an interesting problem.. When using the bidirectional 3G SDI/HDMI adapters sound comes in REALLY hot, and via all of my adapters, not just one.
What do I mean? If I have a laptop connected to the adapter via HDMI, I have to attenuate the audio down to under 40 (out of 100) in Windows are I get horrible distortion. Same issue with other devices such as android phones, and then due to Apple's decision to not allow any sort of attenuation when connected, iPads, iPhones, macOS devices just blast.
It reminds me a lot of accidentally plugging in a LINE device into a PHONO input that has a pre-amp.
Are the SDI adapters just not going to be useable for a stable, non amplified signal?
Thanks!
What do I mean? If I have a laptop connected to the adapter via HDMI, I have to attenuate the audio down to under 40 (out of 100) in Windows are I get horrible distortion. Same issue with other devices such as android phones, and then due to Apple's decision to not allow any sort of attenuation when connected, iPads, iPhones, macOS devices just blast.
It reminds me a lot of accidentally plugging in a LINE device into a PHONO input that has a pre-amp.
Are the SDI adapters just not going to be useable for a stable, non amplified signal?
Thanks!