I might be especially clumsy, but I wouldn't want to ride without those heat shields.
I routinely touch them and am happy that the damage is usually minimal or only a reminder to not put your leg there ;)
EDIT: I'm also quite small, that might be a factor too.
Cheers,
D.