ActionScript and Java – problem with rounding

There was a big thread on FlexCoder forum related to the lack of BigDecimal class on the client. It boiled down that the only implication of that is that if you use BigDecimal and other exotic datatypes on the server side it can be rounded incorrectly.  To cure it there was proposition to build custom ActionScript BigDecimal class and serialization that would solve the issue. Here are my thoughts on the subject – in the thread context:

There are few ways to look at the problem – theoretical (as a former mathematician) or practical (as Wall Street programmer for the last 17 years). It is a big problem – theoretically. Numbers obviously do not match. It has always been that way – and I can probably dig messages from 15 years old messages CompuServe describing exactly the same precision issues with almost any type (timestamp, numbers, rowid, etc) when the client and the server do not share the same underlying data structure. The practical solution I have seen is not to do calculations on the server, but rather do intelligent rounding on protocol level. With your sample of x=1.919999… rounding to 2 decimal places you go with x = round( round(x+.5),3),2)

Please take a look how the db drivers deal with the same issue when they need to include timestamp or double in WHERE portion of update/delete statement – they always have the same issue as client precision is almost always different from the server one.

If you use automatic code generator for your database access ( we use our own daoFlex tool that you are welcome to take a look at), you can use DB precision attribute to automate rounding for the numbers on updates – making your programmers blissfully unaware of the problem.

The next step is to understand why Flex is dealing not just with single double type, but gets “int” in the conversion. Typical DB case would be retrieval argument. If you try to update Sybase’s table that have int id as primary key, the statement “where id=?” with setDouble(1, param) it will not use index on id (but will find the record as the rounding will kick in later). At that point you are looking at full table scan to update single record – and it might be much worse problem then loosing one cent on rounding.

Bottom line is that you always have to be aware that ANY data crossing machine boundaries with an exception of string needs to be rounded. You have to make sure your DAO layer can make use of DB metadata to hide it from the developers that are guaranteed to make the mistake.

Anatole Tartakovsky

One thought on “ActionScript and Java – problem with rounding

  1. I agree with all you said, but your statement on losing one cent.
    Losing one cent because of rounding is the a very serious problem, and if the table can is your only solution, go for it.

    Let me tell you a real story that happened about 20 years ago. There was a programmer, with a weird last name that started from Z, and he was always the last person in any lists or rosters. He decided to add to the salary calculation program a little if-statement, that would put all rounding leftovers to whoever is the very last surname.
    He was making money for some time, until a lady with another Z-last name was included into the database. She started noticing some extra money deposited to your account and started to ask questions…This guy is not with us anymore :)

Comments are closed.