with the events of the movie, it's a possibility that before it, Princess Twilight might've warn all three ponies races to stop fighting amongst one another, and just live in harmony, and obviously they likely didn't care to listen, so she and possibly along with other ponies that she trusts(some might've been the ancestors of the mane 5 (g5), which would probably explain why argyle had the earth pony crystal), to use the crystals by taken ALL the magic from them, cause if they choose to forget what the magic really means, then the pony races don't deserve any of it.

note: this is only speculation, this may not be true to the events that happened before the g5 movie, until some confirmation from Hasbro.

and also this video got me thinking about what had happened before.

update: I adjusted the text there