There is also a metal disk in there, I assume to help keep clutch dust out of the armature. I have seen that disk come loose and rattle around a bit when you crank the motor.
Either way, the starter needs pulled apart and disassembled. You can get rebuild parts for it.