April 10, 2008

math precision out your ears

we very often have to deal with scientific data that's been gathered by various sensors or lab equipment, some of it needing math done with a fairly high precision that ColdFusion sometimes couldn't handle. flopping down to java BigDecimal was the usual way of dealing with this. but yesterday in the ColdFusion support forums, Hemant Khandelwal causally mentioned that cf8 had added a new method, precisionEvaluate() for just such cases. i guess it was one of those "senior momements" that made me skip right past this puppy when i was reviewing the new stuff in cf8. in any case, "wow".
<cfscript>
// the old way if you needed the precision bd=createObject("java","java.math.BigDecimal");
bd1=bd.init("234.567890123456");
bd2=bd.init("123.67141525678901345");
x=bd1.multiply(bd2);
// "high" precision math y=precisionEvaluate(234.567890123456*123.67141525678901345);
// "normal" cf math
z=234.567890123456*123.67141525678901345;
writeoutput("<table border='1'>
<tr><td>old fashioned java way</td><td>#x#</td></tr>
<tr><td>new precisionEvaluate</td><td>#y#</td></tr>
<tr><td>normal cf math</td><td>#z#</td></tr>
</table>"
);
</cfscript>

which produces something like:
old fashioned java way29009.34294536678530209026494448320
new precisionEvaluate29009.34294536678530209026494448320
normal cf math29009.3429454

i would imagine that cf's handling the expression parsing and data type casting in the background similar to the above example using BigDecimal.

ps: coldfusion 8 added BigDecimal as one of the data types for javaCast(), so you could ditch the createObject() and use something like this:
zz0=javacast("BigDecimal","234.567890123456");
zz1=javacast("BigDecimal","123.67141525678901345");
zz=zz0.multiply(zz1);

0 Comments:

Post a Comment

<< Home