Come on. As stated in the manga, the only Uchiha he was shown to be frightened of is EMS Madara:
And EMS Madara is right under Hashirama, who was stated to have made the five current Kage's combined efforts look like nothing, and be above the likes of DSM Kabuto. Who wouldn't be wary of another EMS Madara appearing? Seriously.
Even after Sasuke displays his Eternal Mangekyou Sharingan, Tobirama still refuses to respect him, that must say something in and of itself (although Tobirama believes it only to be MS):
The assumption I can make from this is that your-average-MS-user isn't up to par with Tobirama (once again, because Tobirama isn't aware that Sasuke actually has EMS).
Yes. Orochimaru taunts Tobirama by saying he's frightened of Madara, and Tobirama replies by saying that Orochimaru knows nothing about Madara. Basically giving reasoning as to why he was frightened of Madara.
I still do believe him to be fearful (not necessarily afraid) of EMS Madara. Nobody else though, because the only people we know to be superior to Tobirama are EMS Madara and Hashirama, who are Top 5 at the very least (and Top 2 at the very most, if you're not counting SO6P).