I've enjoyed all the Saints Row stories too. Volition's sense of humor has always been very enjoyable to me, and that's one big thing that set's it apart from it's arch nemesis -- Grand Theft Auto. As Minimaul said, Gat Out of Hell is a standalone expansion, so I wouldn't have the same expectations for it as I have for the full Saints Row games.
I think much of the "hate" for the newer games stems from the fact that many of the "sandboxy" open world features that we loved in the older games have not been included in newer games (particularly, Saints Row 4). I would expect that, since Volition has such a huge arsenal of work to build upon, they would use that as a baseline for the newer games and then build upon it.
Instead, what I see is many of the features that attracted me to the Saints Row games being removed in the newer games. It seems like Saints Row 2 was the pinnacle of Saints Row games, as far as the open world goes, and the open world has been getting less interesting for each game. I do like Saints Row 3 for it's BEAUTIFUL graphics, but Steelport is still not nearly as interesting as Stilwater. Saints Row IV was a particular disappointment, since it simply took the Saints Row 3 world, destroyed almost all of the interesting buildings, removed weather, boats, cribs, etc.
Again, I enjoyed all the stories, so that's not my objection. But when the campaign is over in Saints Row 2, for example, there's still this huge and vibrant world to enjoy with all sorts of fun things, and that's what I loved about the Saints Row games.
So, I think the "hate" is not really hate but rather disappointment that the older games just have much more playability than the newer ones and are just more vibrant than the new ones.