The only other issue is a kind of buzzing noise in the background when I use a headset.

Bought this for my business but it has a high pitched buzzing noise that is noticeable and it also caused the other phones to have the same high pitched buzzing noise

It seems that this phone is unaturally amplified which also amplifies your own voice heard loudly in the earpiece as well as all the buzzes noises, hisses and clicks of the telephone line