There are several methods to reduce corruption.
Build law-increasing buildings (temples that increase law and eastern civs have the execution square); there are mods that give other buildings some law increasing as well.
Look for governors with good law increasing treats and are not too corrupt. If you don't have good governors keep them out of your cities.
Place your capital in a central position of your empire or even better in a central position to your cities that generate the most money. The distance to the capital increases unrest and corruption. This last bit is not well known. The first thirty squares from your capital are free from this source of corruption. If you increase the distance to more then 30 squares the corruption goes up in a linear way with a maximum of 80%. (If you ever tried to block a path with a fort you will have noticed that the fort is on a grey square. The total strategic map consists out of these kinds of squares. These are the squares I'm talking about.)

Friesian

Corruption is one of the factors in capitol location & placement.

According to research done by therother, here are some rules of thumb concerning capitol cities:

There is no penalty within 15 squares of your capital.
The penalty is always 80% over 86 squares away.
The penalty increases by ~1% per square.
So between 15-85 squares, take the number of squares, subtract 10, and round to the nearest 5. So 24 squares would be 24-10=14, so the penalty is 15%. There seems to a small aberration near the transitions though.

Online Tools for computing optimum capitol location vs. penalty:
by d0t
by Ravenous Bugblatter Beast (advanced verion of d0t's)

This thread is related to this one: Noob Question... :'( [change the capitol/captital city]

This thread now Indexed as:

Corruption, capitol [capital, distance, money, denarii, loss]: 1 2

thanks alot, i'll give that a go

I've run into this. I found that I was exterminating every settlement and the preceding ones I own were going in the shitter...so I stopped doing that as often and managed to maintain a stable economy with the entire faction.