Because Starks were the Usurper's dogs, a volatile and foul breed of mankind, the sort that would definitely treat their charges with every horror just to amuse themselves. Or...so he'd built them up as, in his head. A mix of paranoia and rage with healthy doses of encouragement from various sources across the years made for enemies far more interesting than their realities.

"I sincerely hope that he does." It would take far more than meeting a few who didn't seem like horrible human beings for Viserys to change his mind on that. Too long in exile, too long left with his mind to wander and rot in furious humiliation. "One day...Lannister, Baratheon, Stark, Arryn...they all must fall."

Uh yes he does. TFW you wanna kill people for Daddy but your sister does everything better. Sigh.