Is going to a specialist my right? [U.S., MI]

I was diagnosed as Type 2 just over 1 year ago. I was told "it's not severe, but it's definitely Type 2". My A1C was 7.5 and non-fasting sugar was about 250. Ever since going on Metformin my A1C has been 5.8-6.3 and my sugars are down to around 120.

At my most recent visit, my doc put me on an ACE inhibitor Lisinopril, 5mg, to help with what he called Kidney scarring or something (I know, I need to start taking notes when my better half doesn't come with me). After that appt, I was freaking out a bit, with the addition of new meds, and a friend recommended me to an Endocrinologist they've been really happy with. So I called the specialists office, and I need a referral. Called my doctor to get a referral, and was told "we only refer to specialists in cases where the diabetes is more severe or we feel the patient isn't handling it well" and was denied.

Am I allowed to fight back with them and demand a referral to an Endocrinologist? Or should I go to a different MD and get them to refer me? I really like my current doctor, but feel no matter how "serious" my diabetes is, I should be able to see a specialist about it.

Am I being paranoid?