Pepe is used by the entire internet, the only thing that links it to the alt-right with greater relevance is the insistence of the media.
But if you really want to go that way, we can also call Animal right's legislation and certain worker's rights legislation, Nazi legislation I guess...
Hitler liked dogs, do you like dogs? You have Hitler tastes I guess...
Don't be an idiot. No, Pepe is not used by the entire internet, just a few messageboards. In fact, I personally hadn't seen much of him (apart from him starring in a few memes) until this whole alt-right stuff started. I imagine that the same is true for a lot of people who are not clued into the whole chan meme culture; right now, it's the alt-right that make Pepe popular, not anyone else, thus calling him an alt-right icon (and I repeat myself here) is not wrong.
I made the comparison to the swastika very deliberately. It too was a symbol that (depending on culture) had a number of different meanings (with most of them obscure), until one group appropriated it for their purpose. A similar thing is happening with Pepe and the alt-right: While there's a large corpus of memes featuring that particular frog, none of them ever leaked out of the internet image board bubble. Now that a particular group inspired and formed by that bubble (the alt-right) are making their appearance on a larger stage, they are dragging their iconography (of which Pepe is a part) with them. As a result, for a lot of people, Pepe and the alt-right are now associated with each other. Claims that they shouldn't be associated because that's not what Pepe is about are ... well, they're a brand of nerdery that is irritating at best.
Symbols do not have their meaning set in stone when they are created. They change in accordance with their use, and how that use is perceived by the wider world. Once the media started to associate Pepe with the alt-right, that association became fact for a lot of people. Whatever he was before stopped mattering, just like it doesn't matter today what the original meanings of the swastika were. So, just like the people who were using the swastika correctly had to stop doing it lest they be associated with something outside of their control, you have to give up this idea that Pepe doesn't have a political meaning. He does, now. End of story.
Oh, and about that whole animal rights and workers rights thing: Please learn how to hyperbole. You aren't doing a good job of it right now. Your example with the red cross shows this: The red cross, just like the american flag, is a well-known symbol of something very specific. If anyone wants to change the meaning of it, that person has to be very very powerful and use a long campaign to do it; Pepe, as something completely unknown, doesn't need that sort of power to redefine his meaning.