Recall that when he first escaped from Azkaban, within months he had enough wits to steal a journal and start producing inciting columns giving instruction in magic, among other offences.

Azkaban seems not to have affected him as it would any other wizard. Suspect it has to do with the man's utter lack of pleasantness in the first place; there was very little room for him to slide into rage or despair when he wallowed in them all his days.

And he clearly had an accomplice. An insider to the Ministry.