Hmm... I think the term Ami developed around the US military bases in Germany, yes. I grew up near one of the biggest of those bases, and the term's very much in use there, while you don't hear it that often in other parts of the country.
But I have heard people say Ami when they simply meant a US citizen, and also, in Bavarian bei die Amis (this is a mistake in standard German!) means "in the US".
By the way, in case you're wondering about pronunciation: in German, Ami rhymes with "mummy". 
It's sweet of you to put my opinion above the Duden's, Steven, but you do know that it reflects usage, don't you? So, it's quite possible that Ami used to be a term for US military personnel 30 or 40 years ago, but now refers to any US citizen. 
Thanks Alharacas, for your comments.
Coincidentally, today I was able to ask a local German native speaker this question at lunch. She and her husband run a local German bakery they started here 26 years ago. I always like running into her there because I can speak German with her.
I asked about Ami,and she confirmed that it just means an American of any sort, and not just soldiers. I told her it was not a word I ever use, and really not even one I’ve heard people use much in Germany today. Plus, I told her I wasn’t really too fond of the word because I felt it had a slightly abwertend, pejorative, sound to it. Not terribly pejorative, but only slightly so. She agreed and said she doesn’t like the word either, and laughed and said her relatives will often email her and tell her she’s in Amiland. So, we got a laugh out of that.
Duden doesn’t always give a lot of details on the history of words, but I strongly suspect Ami originated in the late 40s to 50s and did refer to American soldiers. Then, as the decades passed, the word probably changed (like words do with time) and came to refer to all Americans.
Like I said, I would swear I heard it said no later than around 1980, that the word referred to American soldiers, and that may have come from one of my German professors at the time, most likely the native speaking lady from Hamburg. But, she was 60 at the time, and had been living in the US for at least 20 years by then, and would have been speaking a German used primarily in the 1940s and 1950s. When I think of Amis, I immediately think of post-war occupying American troops, which is why it sort of has a slightly negative feel to it, like the British use of yanks.
I was a bit surprised at how German has undergone some changes in the last 4 decades. I’ve actually heard native speakers say „in 2018“ , which is totally wrong, as you know. When I learned German I was taught you NEVER said that. It was either „im Jahre 2018“ or just „2018.“ Take your pick, but you can’t say „in“. Now, I heard someplace a few years ago that saying „im Jahre“ is a little old-fashioned sounding and that the „e“ in „Jahre“ can and often is omitted. Leaving it off sounds weird to me, so I just leave it in.
Oh well. It’s just one of many examples of how languages can change a little in just a generation.