It's not so much the Falmer that are the problem, but the Chaurus (chauri?) that they control. I'm kind of regretting not having made an Argonian character first. That poison resistance would be extremely useful.
The Falmer armor is kind of cool, though. The helmets are particularly interesting.