BigDecimal and your Money

I often see Java developers attempting to use the BigDecimal class to store monetary values for financial applications. This often seems like a great idea when you start out, but almost always comes back as a flesh-eating zombie intent on devouring your entire time some time later on. The BigDecimal class is designed around accuracy and has an almost infinite size. However: this choice will almost always come back to bite you, or someone else attempting to find a minor bug later on. It’s an especially bad idea when it comes to banking applications. So why is BigDecimal not well suited to storing monetary values then? For one thing: it doesn’t behave in a manner that is practical for financial systems. While this statement flies in the face of everything you’ve probably been taught about this class, it’s true. Read-on and I’ll tell you why.

Theres more numbers in there than you think!

A BigDecimal is not a standard floating point number. Instead: it’s a binary representation of a number. This means: 0.0 != 0.00. While this doesn’t seem like a problem at first: I’ve seen it cause no-end of strange little bugs. The only way to accurately determine whether two BigDecimal objects have an equal value is by using the compareTo method. Try these two little unit tests:

@Test
public void testScaleFactor() {
    final BigDecimal zero = new BigDecimal("0.0");
    final BigDecimal zerozero = new BigDecimal("0.00");

    assertEquals(zero, zerozero);
}

@Test
public void testScaleFactorCompare() {
    final BigDecimal zero = new BigDecimal("0.0");
    final BigDecimal zerozero = new BigDecimal("0.00");

    assertTrue(zero.compareTo(zerozero) == 0);
}

This technique works when you’re in control of the data and the comparison, but it breaks when you want to put a BigDecimal object into most other Java data-structures. I’ve actually seen someone use a BigDecimal as a key to a HashMap, which of course didn’t work. The solution in this case was simple: change the HashMap for a TreeMap and things were happy. However it won’t always be this simple.

They’re true high precision structures.

This doesn’t just mean that they are precise, it also means that they won’t run any calculation that wouldn’t result in a representable answer. Take the following code snippet as an example:

@Test
public void testArithmatic() {
    BigDecimal value = new BigDecimal(1);
    value = value.divide(new BigDecimal(3));
}

Primitive numeric types would just swallow this and represent the 0.3* as best they could, while a BigDecimal throws an ArithmeticException instead of attempting to represent a recurring number. In some cases getting an error will be desirable, but I’ve actually seen someone resolve the ArithmaticException like this:

try {
    return decimal1.divide(decimal2);
} catch(ArithmaticException ae) {
    return new BigDecimal(decimal1.doubleValue() / decimal2.doubleValue());
}

Yes folks, unfortunately I’m quite serious here. This is the sort of bug introduced by an error occurring, computations stop running, and someone adds a “hack” to just “make it work quickly and we’ll fix it later“. It’s a total disaster, but I see it far to often.

They don’t play nice with Databases.

According to the JDBC spec database drivers implement a getBigDecimal, setBigDecimal and updateBigDecimal functions. They seem like a great idea, until you ponder that your database may not have a suitable storage type for these values. When storing a BigDecimal in a database, it’s common to type the column as a DECIMAL or REAL SQL type. These are both standard floating-point types, with all the rounding errors that implies. They are also limited in capacity and will often overflow or cause a SQLException when attempting to store very large BigDecimal values.

The only practical solution which will keep all the BigDecimal functionality and accuracy in a database is to type the amounts a BLOB columns. Try to imagine the following table structure if you will:

CREATE TABLE transactions (
    initial_date DATETIME NOT NULL,
    effective_date DATETIME NOT NULL,
    description VARCHAR(30) NOT NULL,
    source_id BIGINT NOT NULL,
    destination_id BIGINT NOT NULL,
    in_amount BLOB NOT NULL,
    in_amount_currency CHAR(3) NOT NULL,
    effective_amount BLOB NOT NULL,
    effective_amount_currency CHAR(3) NOT NULL,
    charge_amount BLOB NOT NULL,
    tax_amount BLOB NOT NULL
);

That required four different BLOB columns, each one of which will be stored outside of table space. BLOB objects are very expensive both to store, and to work with. Each one often uses it’s own database resources (much like an internal cursor) to read or write the value. This translates to much more time and network usage between your application and it’s database. To add to the misery a BLOB is generally not readable by a SQL tool, one of the major reasons for sticking with a SQL database is that it can be managed from outside of your application.

Performance.

This is often raised as an issue, but ignored in favor of “accuracy”. The performance of BigDecimal is often considered “good enough” for general computing, and it’s fine if you want to add tax to an item every once in a while, but consider the number of interest calculations per month a moderate sized bank do. This may seem like an extreme case, but if your application ran a simple shipping and tax calculation for items on an online store in a JSP you’ve got effectively the same problem. In a very simple multiplication test BigDecimal performed over 2300 times slower than a simple long value. While this may only be milliseconds per mutation, a performance-factor of this size very quickly adds up to more computational time than is actually available to the system.

Also remember that BigDecimal (like most Number subclasses) are immutable. That means every calculation requires a copy of the existing BigDecimal. These copies are generally cleaned away by the eden-space collector (and G1 is very good at handling them), but when you put such a system into production it leads to a massive change in your heap requirements. Your BigDecimal objects must be allocated in such a way that a minimum number of them survive a garbage collection, the memory requirement of such a space quickly spirals out of control.

To add to the performance argument: the compareTo method is quite a bit slower than the equals method, and gets significantly slower as the size of the BigDecimal increases.

A Cure to BigDecimal Woes:

A standard long value can store the current value of the Unites States national debt (as cents, not dollars) 6477 times without any overflow. Whats more: it’s an integer type, not a floating point. This makes it easier and accurate to work with, and a guaranteed behavior. You’ll notice that several different behaviors in BigDecimal are either not well defined, or have multiple implementations. That said: depending on your application you may need to store the values as hundredths or even thousandths of cents. However this is highly dependent on your application, and theres almost always someone who can tell you exactly what unit the business works in. Bare in mind also that there are often de-facto (or even mandated) standards which exist between businesses about what unit of money they deal in, using more or less precision can lead to some serious problems when interfacing with suppliers or clients.

The mechanism I generally try to use is a custom-built MoneyAmount class (each application has different requirements) to store both the actual value, and it’s Currency. Building your own implementation opens the opportunity to use factory methods instead of a constructor. This will allow you to decide on the actual data-type at runtime, even during arithmetic operations. 99% of the time, an int or long value will suffice – when they don’t the implementation can change to using a BigInteger. The MoneyAmount class also enables you to define your own rounding schemes, and how you wish to handle recursive decimal places. I’ve seen systems that required several different rounding mechanisms depending on the context of the operation (currency pairs, country of operation and even time of day). For an example of this kind of factory discussion: take a look at the source-code for the java.util.EnumSet class. Two different implementations exist: the RegularEnumSet class uses a long to store a bit-set of all the selected constants. Given that very few enum values have more than 64 constants this implementation will cover most cases, just like a long will cover most requirements in a financial system.

Summary

This post is to warn people who are busy (or about to start) writing a system that will run financial calculations and are tempted to use BigDecimal. While it’s probably the most common type used for this purpose in the “enterprise” world, I’ve seen it backfire more times than I care to recount. My advise here is really to consider your options carefully. Taking shortcuts in implementation almost always leads to pain in the long-run (just look at the java.util.Properties class as an example of this).

About these ads

10 Responses to “BigDecimal and your Money”

  1. ignorante Says:

    If your database does not have true decimal types (as old mysql versions do) then use a proper DB. Some numeric computations will happen in the DB and the rounding error will appear sooner or later.

    Storing BigDecimals as blobs prevent queries like this

    select * from employe where salary > 1000

    which might be a real performance problem for example when filtering in the application.

    Implementing all rounding and other operations using a long might be ok if the performance improvement justifies spending many hours coding and debugging that code compared to just using BigDecimal and forgetting about it.
    Besides if your coding team does this:

    new BigDecimal(decimal1.doubleValue() / decimal2.doubleValue());

    I think it would be easier to teach them to use BigDecimal properly than to teach them to code a long based decimal math library from scratch without bugs.

    If you run java on one of those big iron servers using Solaris, then there you can access hardware accelerated bigdecimal coprocessors.

  2. Alex Says:

    Some good points! I think what your really highlighting is the need for encapsulation. Using encapsulation you can:
    – Hide the peculiarities of the BigDecimal implementation (e.g. using ‘compareTo’ instead of equals, set some upper and lower bands).
    – Change you underlying implementation to be integer based (or something else) solution as you describe with out impacting the functionality your providing.

  3. Observateur Says:

    Whats wrong with java.util.Properties Oo? Plz open my mind ;)

    • Jason Says:

      It’s not the use of java.util.Properties that is a problem, but rather it’s implementation. Properties extends Hashtable, which is a quick-and-dirty implementation especially when you consider the presence of the Dictionary class. Properties logically not a Hashtable, it’s a key-value map for String objects with load and save functionality. The extension of Hashtable is an implementation detail that as a user of the class you shouldn’t be aware of.

      Given that a few years after Properties was implemented the Collections Framework became a standard part of Java: the fact that Properties extended Hashtable became a problem: it couldn’t be changed to directly implement Map or extend AbstractMap.

      When generics were added in Java 5, Properties already had a long history and now extends Hashtable rather than the more logical AbstractMap.

  4. MoneyField // JavaFX News, Demos and Insight // FX Experience Says:

    […] blog I found talking about the issues with BigDecimal for representing Money is this one titled BigDecimal and Your Money. I think this same technique that I’m going to show works equally well whatever the data type […]

  5. Boris Says:

    RE:A BigDecimal is not a standard floating point number. Instead: it’s a binary representation of a number.
    Please rephrase it because *everything* in computers has binary representation – it sounds really weird, BigDecimal is about representing *decimal numbers* and it’s no matter how it’s stored in computer memory.

  6. Noah Yetter (@angrynoah) Says:

    DECIMAL / NUMERIC / NUMBER database types are absolutely NOT floating point types, not sure where you got that idea. These are fixed-point arbitrary-precision types just like BigDecimal. And while they are limited in capacity, that capacity is typically stratospheric. Oracle’s NUMBER will store up to 38 base-10 digits. PostgresQL’s NUMERIC will store up to 131072 base-10 digits.

    If you are using BLOBs for numeric values you are Doing It Wrong.

  7. Decision Making - Page 3 Says:

    […] does this because scaling integers is easy and correct. So that led to some googling, and I found a page with a most interesting fact: "A standard long value can store the current value of the Unites […]

  8. DS Says:

    “When storing a BigDecimal in a database, it’s common to type the column as a DECIMAL… these are standard floating-point types, with all the rounding errors that implies.”
    …Like everything in SQL, this is implementation-dependent, but no, this is usually not true. MySQL, for example, stores these as exact values, with the number of decimal places you specify when you create the column. (MySQL limits the overall number of decimal places to 65, but if we’re talking about money and 2-decimal-point accuracy, that is 1 sexdecillion trillion dollars).

    So, the complaint about databases is just false. The rest, OK, that’s an opinion, but it has the feel of a rant. For example, yes, when dividing you need to take care to specify what to do about precision and rounding. But that’s kind of the point. You have to do that because you’re using money and you have to have a policy of what happens when you divide and the result doesn’t fit into cents.

  9. Alex Glazkov Says:

    Good summary of the BigDecimal flaws, although, I think several issues are conflated. BigDecimal rightfully throws an ArithmeticException when you try to divide 1 by 3 with the unlimited precision. You should limit the precision (e.g. do something like BigDecimal bd13 = new BigDecimal(“1.0″, MathContext.DECIMAL128).divide(new BigDecimal(“3.0″, MathContext.DECIMAL128), MathContext.DECIMAL128);)


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

%d bloggers like this: