The only reason the Thalmor even came to power in the first place was due to the chaos caused by the Oblivion Crisis (and their successful lie in their home territory that it was them who resolved the crisis). There are quite a few High Elves who dislike them for their extreme ways. The number of non-Thalmor in Skyrim who support them can be counted on one hand, and even those seem mostly to be paid-off agents.

The Thalmor, the fascist High Elf government.

The reason for this hatred is because he surrendered to the Thalmor, but when you meet him he comes across as a supremely Reasonable Authority Figure, as well as a perfect example of Face Death with Dignity, so the hatred seems somewhat undeserved. The very one who hires the Dark Brotherhood to have the Emperor killed is a member of the Elder Council. Even those supposedly loyal to him are more loyal to the Empire as a whole than they are to him.

While not the victim of a effigy-burning ritual like Olaf (possibly because he's still in power), very few people, if anyone at all, seems to have anything positive to say about the Emperor.

The Emperor himself, Titus Mede II, though to a lesser degree than Olaf.

(However, that may be because she's an Imperial.) Also unlike Olaf, her ghost is not found in Sovngarde, either before or after the two quests concerning her, so there truly is no silver lining with her. Apart from some crazy necromancers trying (foolishly) to bind and enslave her, and some vampires living in her catacombs, no one has anything positive to say about her. Unlike Olaf, however, there is absolutely zero dispute over how evil and how reviled she was.

And speaking of long dead reviled rulers, Wolf Queen Potema.

He's also actually the same person as Olaf One-Eye, who has a 100% Heroism Rating, meaning he got both a Historical Villain Upgrade and a Historical Hero Upgrade at the same time. He seems to have had a better approval rating in the other Holds, but his reign was so long ago that it is hard to say for certain whether it was good or merely not bad enough to inspire annual effigy-burnings. Solitude has a festival where he is burned in effigy.

Skyrim creation kit magic effect