The rich were taking advantage of the Poor. Previously outsiders like Sanballat and a company of hooligans created problems. Now the problem has come from within (which is always harder to solve). 

Earlier on, the first internal crisis had to do with too much to do but too little people to help.  "The strength of the burden bearers is failing, and there is too much rubbish so that we are unable to work on the wall" (4:10). In other words, there was lack of human resources. Nehemiah solved the problem by adding reinforcement (cf. 4:13).