On my Norge, at about 90,000 miles the buttons had become loose enough that the left outer rotor could be heard 'clacking' when I pushed the bike without the engine running. Once I figured out what the cause was, I did a brief test brake scenario, enough to activate the ABS - it was noticeable that the rotor had too much play.
Have a dealer check it, but if you can hear the outer rotor move on the buttons it's definitely time to replace them.