Globalizing ColdFusionMX: 02/01/2005

rokuyo

i seem to have datetime on the brain this month. one of the trickier things i've been trying to get a handle on was how to calculate japanese "rokuyo". what's "rokuyo"? well, let me tell you....

a lunar calendar was used in japan from the 14th to the 19th century. that calendar had a six day week and those six days were known as rokuyo. and like any other calendar system, each day had a name and a particular meaning (you do know that the english weekdays are named after one of the seven "planets" of ancient times?). and of course, each day had a significance:

sakigachi good luck in the morning, bad luck in the afternoon
tomobiki good luck all day, except at noon
sakimake bad luck in the morning, good luck in the afternoon
butsumetsu Unlucky all day, as it is the day Buddha died
taian 'the day of great peace', a good day for ceremonies
shakku bad luck all day, except at noon

source

while i'd guess few people would admit to closely adhering to this system, it does invoke some strange "better safe than sorry" behaviors. for instance, some hospital patients in japan won't agree to be discharged on butsumetsu day, as it's regarded as being very unlucky. rather they'd stay the extra 24 hours to be discharged on a lucky taian day.

the calculations for determining rokuyo turn out to be surprisingly difficult. in fact, the only published code i ever saw for this was developed by Eirik Rude, a cf developer (at that time living in japan). the complexity comes from the need to calculate lunar months (remember the old japanese calendar?). since i wanted to integrate this functionality with our existing icu4j-based calendars, i poked thru the lunar calendars (chinese, islamic and hebrew) that i knew about to see if we could use any of these. of course, the old japanese lunar calendar was basically the lunisolar chinese calendar. using Eirik's basic logic and the icu4j library i was able to considerably reduce the code's complexity (the complexity's still there, but i pushed it down into the icu4j java library where smarter people than i have already dealt with it).

the rokuyo testbed is here and the i18n calendars package incorporates this new functionality (pick japanese calendar from the select). and this is a good resource if you want to read more about rokuyo.

gonzo journalist gone

CNN's reporting that one of the icons of my youth, hunter s. thompson, committed suicide at his home in colorado on sunday.

oh my.

universal time

all this poking and prodding into cf's datetime i did lately shone a bright light on the usefulness of something like icu4j's universal time class. if you have to swap back and forth between time scales (for instance some java classes require a long instead of a date type) or even if you do "simple" date manipulations (say averaging two java datetimes could cause overflow even with current dates), you've got good candidates for using universal time. to make a long story short, i built a universalTime CFC to help handle this. below is some output from this CFC (we'd normally have a testbed on our site but for some reason this class won't load via spike's remote classpath technique):

time:= {ts '2005-02-20 16:56:03'}
cf epoch:=38403.7055903 (days since 31-dec-1899)
universal time from cf time:=632,447,745,630,000,000
universal time to cf time:= 38403.7055903

coldfusion timescale:=38403.7055903 (days since 31-dec-1899)
excel timescale:=38403.7055903 (days since 31-dec-1899)
db2 timescale:=38403.7055903 (days since 31-dec-1899)
windows timescale:=6.3244774563E+017 (ticks (100 nanoseconds) since 1-jan-0001)
windowsfile timescale:=1.2753478563E+017 (ticks (100 nanoseconds) since 1-jan-1601)
mac timescale:=130697763 (second since 1-jan-2001)
oldmac timescale:=3191849763 (seconds since 1-jan-1904)
unix timescale:=1109005407 (seconds since 1-jan-1970)
java timescale:=1.109004963E+012 (milliseconds since 1-jan-1970)

the CFC will be in the usual places in a bit.

icu4j has moved

just in case you haven't been notified, the icu4j sites have moved.

ICU main page
library's download page
ICU documentation page, with the icu4j API docs now here
icu4j FAQ
RB manager
additional docs

on the topic of icu4j, i knocked off a couple of pages to explore it's new ULocales class (after somebody asked me how many new locales for India and i had no idea). i was surprised by the answer.

if that doesn't surprise you, try the United Kingdom or Ethiopia.

cf datetime numeric units

i goofed again. when we tested the reworked timezone CFC, the test harness randomly tested just timezones but not dates. and by some bizzaro world circumstance, all the timezones it tested didn't use daylight savings time (DST). so the timezone offsets it reported passed 100%. when i was cleaning up the tests today i noticed that and ran the test against a list of timezones that use DST and dates in/out of DST. oops. all the offsets it returned were all the same as the raw offset (sans DST), even when the dates were in the DST period for that timezone. oh my.

trying to fix that blunder led me into the bowels of cf where i had some trouble getting my head around the exact numeric units that cf uses for datetime objects. i was doing a javacast("long",arguments.thisDate) thinking i'd get milliseconds since java epoch to pass into java.util.TimeZone's getOffset method, ie assuming a cf datetime was a java.util.Date. but that snippet actually returns 38399 for today (16-feb-2005). which certainly doesn't look like the number of milliseconds since 1-jan-1970. recalling i blogged something about binary time scales, i whipped out a calculator to see when cf's "epoch" actually started started. 38399 turns out to be the number of days since 31-dec-1899. that makes the cf datetime object type (which is actually a coldfusion.runtime.OleDateTime) more like "Excel_Time" or "DB2_Time" which measure dates in days since their epoch started (31-dec-1899).

well the timezone CFC's fixed up now. and i learned a bit more about cf (and not to assume too much about how cf works).

new and improved timezone CFC

our code re-working frenzy continues. trying to squeeze more speed out of our venerable timezone CFC (circa AD 2003), we revisited each method looking for "dumb things". one thing we noticed was its use of the gregorian calendar to determine a timezone's offset. why we did it that way has long since been forgotten (but probably someone who knew java better than we did told us to, we were looking at using other calendars, the fact that it worked at all was a novelty, etc.).

in any case functions (castFromUTC/castToUTC/etc) that used code like this:
// get selected timezone
tZ=tzObj.getTimeZone(arguments.thisTZ);
// set gregorian calendar to selected timezone
gregorianObj.setTimeZone(tZ);
// set gregorian calendar to selected datetime
gregorianObj.setTime(arguments.thisDate);
// calculate offset
thisOffset=(gregorianObj.get(gregorianObj.DST_OFFSET)/1000) +
(gregorianObj.get(gregorianObj.ZONE_OFFSET)/1000);
return dateAdd("s",thisOffset,thisDate);

were re-worked to use:
// get selected timezone
var timezone=tzObj.getTimeZone(arguments.thisTZ);
// get offset in hours, also considers DST
var thisOffset=
timezone.getOffset(javacast("long",arguments.thisDate))/3600000;
// send cast date back to caller
return dateAdd("h",thisOffset,arguments.thisDate);

and sped things up 2x-3x. you can see the new CFC in action here.

new and improved i18n calenders

i've completely re-worked the individual I18N calendar CFCs. these are now consolidated into one package. most folks using these calendars (at least the ones talking to us) tend to use more than one, so this made some sense, especially as we re-worked the codebase so the 5 non-Gregorian calendars (Buddhist, Chinese, Hebrew, Islamic,Japanese) now extend the "base" Gregorian calendar CFC. our little hand waving at the OO bandwagon currently rolling around CFville. we think it actually will make some improvements in at least code maintenance. previous versions were distributed standalone, with most of the CFC code being duplicated across calendars, the major difference being which ICU4J calendar class the CFC rode. the common functions are now in the base gregorianCalendar CFC, with the other calendars extending that and initializing their own ICU4J calendar class. the codebase went from 7k lines down to 2k lines (and almost half of that being comments).

the code is also considerably improved, its now based on ICU4J version 3.2 and it's ULocale class (232 locales, 100 more than blackstone). several of the more commonly used functions have been re-written and we're seeing 3x-4x speed improvement over the older versions. frankly, i'm a bit baffled why, for instance:

following the ICU4J API and some examples, we initialized date formatting objects with the calendar class (Buddhist, Chinese, Gregorian, Hebrew, Islamic,Japanese) we were working with:
//init calendar with timezone and locale
var thisCalendar=aCalendar.init(utcTZ,thisLocale);
// return formatted date
return aDateFormat.getDateInstance(thisCalendar,tDateFormat,thisLocale).
format(dateConvert("utc2local",arguments.thisDate));

was reworked into this:
// init date formatter object with date format, locale and default calendar
var tDateFormatter=aDateFormat.getDateInstance(tDateFormat,thisLocale);
// swap calendars
tDateFormatter.setCalendar(aCalendar.init(utcTZ,thisLocale));
return tDateFormatter.format(dateConvert("utc2local",arguments.thisDate));

this builds the date formatter object with the default calendar, then we swap it to the calendar we want to use (the tDateFormatter.setCalendar bit). that sped up this function 3x-4x! while it "seems" less efficient it actually worked quite a bit faster.

you can see the testbed and download the CFC package here. any comments appreciated.

the sweet made sweeter

eric mauviere (he built that little flash map widget used to demo our geoLocator CFC), who i think has produced probably the best flash GIS i've ever seen, has updated his geoClip application. if you want to see how a GIS should be built with flash, check out his application.

the sweet is indeed made sweeter.

blackstone locales

maybe i didn't look hard enough but i haven't seen any mention about locales in any of the blogs/articles/etc. concerning the release of blackstone (now officially known as ColdFusion MX 7). ditto during the beta pr period. no idea about why this was but it's sure like hiding your light under a bushel. if you're a g11n developer, Blackstone's going to be a real eye-opener. core Java's locales are now Blackstone's locales. from the measly 20 odd locales in cfmx 6.1, Blackstone gives us 130. the figure below compares locale support across different versions of cf. pretty cool, huh?

and you can now use Java style locale identifiers like ar_AE instead of the "pretty" locale name Arabic (United Arab Emirates), so now it's that much easier to synch up your calls to core Java's ResourceBundle class from cf. and you can buy into all that locale info using the super simple setLocale() function.

of course, as soon as i get what i've asked for after years of asking, i find some new plaything. as you might have read in this blog, icu4j's latest release (3.2) switched to the CLDR's locales, all 232 of them (with 60 more in beta). the graph below compares cf with and without icu4j.

gives you pause, which should i use for locale support? oh my. i'll be revisiting this issue again.

persian calendar progress

we're seeing some substantial progress on getting an ICU4J-level persian calendar. the folks over at HyperOffice have taken the bull by the horns and put some of their resources in iran onto this. for more details you contact drew morris.

Globalizing ColdFusionMX

February 22, 2005

rokuyo

February 21, 2005

gonzo journalist gone

February 20, 2005

universal time

February 19, 2005

icu4j has moved

February 17, 2005

cf datetime numeric units

February 16, 2005

new and improved timezone CFC

February 13, 2005

new and improved i18n calenders

February 11, 2005

the sweet made sweeter

February 07, 2005

blackstone locales

February 05, 2005

persian calendar progress

About Me

Previous Posts

Archives

Links

XML

Location

Other stuff