Midora did not fear a monster like Joa, but from just being in his presence he saw a vast power within Toriko. You're right that he did not say Toriko was stronger than him, but for a person who is essentially invincible to say that Toriko then would have actually injured him speaks volumes. Toriko's latent potential is greater than Midora's, I really don't see it any other way. Midora would not have an easy time against super Acacia, at least not without eating God himself, and even then I think White and NEO are unique and that no other demons present are at their level. I said before that it would be fitting for Red to defeat NEO but I changed my mind because I think this manga is almost done with and for Red to end things would leave the door shut on ever exploring Toriko's strongest demons.