The important things to note, though, is that not only are you not an asshole, you're actually awesome. And anyone who thinks differently doesn't have their head on straight.
I get those comments from people too. Mostly, they come from a brother I no longer speak to. It's easier for those people who don't have healthy lives or who just don't like their lives to blame the bad feelings on an outside force. It is harder for us to kick them in the balls.