1- They start with weak infantry who are very slow and will rout at the first signs of trouble
2- While Desert axemen are great, when compared to other high end infantry, such as roman leagions, the have reletively low stats
3- Same as above for Pharoah's Guards. When compared to other high end spearmen like sacred bands, spartans, armoured hoplite and the like, they are reletively weak
4- The general is a chariot archer, so he can't charge very well, dies easily, wrecks your own liine when he goes throuhg it and doesnt' listen very well
5- Cavalry is really weak. In a regular charge vs anyone Desert Cavalry will lose almost as much as they kill, and the next cavalry aren't good when compared to that of other factions.
6- While horse and camel archers are nice, they lose usefulness once the enemy actually charges as they have little attack or defense and will die right away
Maybe I am just being pessemistic but they don't actually seem tough when i look at them. One quick question to add: Do towers on walls have infinite ammo?