Mail Includes Names of People You've Never Communicated With

I was going to put this under Wishlist, but the forum there specifically says don't discuss existing "features."

Title: Mail Includes Names of People You've Never Communicated With
Reproduction Rate: 100%
Blocker? No
Details:

When I'm looking for someone's name to send them mail, I currently have to scroll through a ton of other names of random strangers -- people I've never spoken to, who months ago sent me a friend request or party invite or guild invite that I promptly DECLINED, because I'm in private mode and even if I weren't I wouldn't have accepted an invite from someone who didn't bother to say hello first. Some of these are people who I know from seeing them around elsewhere that I don't want anything at all to do with. Not only have I never initiated contact with them, but I specifically refused to interact with them when they initiated it. So why are their names cluttering my contact list? There's no way for me to remove their names either.

Steps to Reproduce: In the Mail UI, click Send Mail then review the list of player names in the lower box.
User Specs: Irrelevant