Assuming the Curie point for the magnet's material isn't hit due to heat from the grinding...
I'd bet that the resulting pile would have some peculiar looking physical characteristics, ferrofluid comes to mind.
Even though the north of one chunk and the south of another would be attracted to each other in tendencies of chains, repulsion due to leakage from 'awkward' alignment of the numerous sizes and shapes would probably cause them to work their way into bows (like solar flares), or peaks (like ferrofluid), or maybe a combination of the two?
Couldn't say for sure, never ground one up to find out. But I'd be willing to bet you'd have an interesting looking pile of dust when you're done!
Take pictures!
Steve