Nagato has his weaknesses...for example, he is prone to genjutsu just as any other ninja as long as he has no summons....which could easily be done at the beginning of a fight by an expert genjutsu user like itachi.

Here are 5 ninja that should carry him high diff/defeat him:

Itachi
Hashirama
Muu
Obito
Naruto