May 28, 2006

there is such a thing as timezone hell

and i know because i've been visiting there for the last 2 weeks or so. and yes, it is a few doors down from classpath hell. if you're outside the US but host in the US or if your ColdFusion application deals w/timezones, and accurate datetime data is important you might want to pay attention this.

let me provide a bit of background first.
ColdFusion datetimes are, i think, coldfusion.runtime.OleDateTime. The actual datetime is stored as decimal day offset from the ColdFusion epoch (31-Dec-1899) which is similar to DB2's and Excel's epochs. you can prove this to yourself by doing something like:

<cfoutput>#javacast("double",now())#</cfoutput>


which should return 38864.5102546 for a datetime of {ts '2006-05-27 12:14:46'}. btw this also hints at another way to examine day only differences between dates, ie. you can do int(now()) to ditch the time bits. there's something curious about ColdFusion datetime because you can also treat them as if they were core java Dates. for example

<cfset now=now()>
<cfoutput>#now.getTime()#</cfoutput>


returns the java epoch offset (milliseconds since 1-Jan-1970), 1148707294281 for a datetime value of {ts '2006-05-27 12:21:34'} in the server's timezone. in fact pretty much all of the core java Date class methods (though many are deprecated) will work on a ColdFusion datetime. you can use toString(), toLocaleString(), toGMTString(), setYear(), etc. though if do use these methods you'll have accomodate core java's Date "quirks" such as years being year-1900, months starting at zero (ie january is month zero in core java), as well as handle tricky data type issues (mainly for setTime() which is looking for a long).

now to the heart of the issue.
it's important to understand that ColdFusion references all datetime objects relative to the ColdFusion server's timezone. if you dump coldfusion.runtime.OleDateTime you'll see a field for TimezoneOffset (similarly core java's Date has a TimezoneOffset getter but no setter). something's gotta go there, might as well be the server's timezone data (i'm sure more thought went into this than that but you get my drift). all datetimes are server datetimes as far as ColdFusion is concerned.

if your application must handle multiple timezones, ideally you'd want to store your datetimes as UTC and convert this to whatever timezone you needed. in fact that's the whole purpose of the timezone CFC. all well and good but all datetimes are server datetimes as far as ColdFusion is concerned. while your intention is that all datetimes are UTC, your ColdFusion server doesn't see that and dependng on your server's timezone, strange stuff can happen. the clearest example is what happens to datetimes on the cusp of Daylight Savings Time (DST) on ColdFusion servers in timezones that use DST (like most of the US-based ColdFusion servers). a datetime of 2006-04-02 02:01:00.0 on those servers would never exist, it would automagically get flopped over to 2006-04-02 03:01:00.0 because ColdFusion doesn't see your UTC timzone only the server's timezone. so for an hour or so twice a year, your datetimes would be wrong by an hour. oops.

lets review some potential workarounds.
  • by far the easiest solution to this problem is to simply set your server's timezone to UTC and bob's your uncle. no fuss, no muss. however i think this won't be a viable solution for many applications hosted on shared servers, so lets look at other workarounds
  • a slicker solution would be for Adobe to provide a setTimeZone() function similar to setLocae() that would force the ColdFusion server to use the selected timezone for all it's datetime operations. if you agree with me on this by all means let the ColdFusion team know.
  • a messier but certainly more portable and near term ready approach would be to use an epoch offset instead of a "real" datetime. two solutions that come to mind are to convert all your datetimes to java epoch offsets as described above or icu4j's universal time scale. right now this is the approach we're going to implement in the new version of the timezone CFC.


i'd like to thank wayne bianca for starting this research in the first place (though given what i've been doing for the last two weeks, i think ignorance is bliss) and spike for helping me stop overthinking the solution.

May 16, 2006

BoardFusion's i18n bits

just in case you missed it, the BoardFusion project (BF) has released a preview of the user interface (UI). and while the project accumulated a lot of useful comments, i'm posting this to solict some specific i18n feedback before we close the books on the UI preview. so this is the "last gas for 200 km" review.

to recap:
  • i18n is a zero level goal (that is the project won't leave home without it).
  • it will be based on icu4j java library and by based i mean every single i18n function, except some parts of the resource bundle CFC and (probably) the Gregorian calendar will be derived from it.
  • besides the basic Gregorian calendar most ColdFusion developers are familiar with, this project will also include Buddhist, Chinese, Japanese, Islamic, and Hebrew calendars to handle that tricky calendar math.
  • user centric timezone, users will see datetimes in their individual timezones--and yes, even this functionality will come out of icu4j. by divorcing this functionality from core Java, the project will be able to take advantage of icu4j's more frequent updates.
  • locale based collation (sorting).
  • strict use of resource bundles (rb), you will be able to l10n skin this puppy, though we haven't 100% decided on the "recommended" rb management tool yet. besides icu4j's rb manager, any ideas?
  • standard localized date/numeric/currency formatting, all hail CLDR.
  • the project will make use of the super cool JavaLoader in order to load the icu4j from off the server classpath (shared hosts will not be a problem). this also allows for painless updating of the icu4j jar file.


so, have we missed anything? some i18n related functionality we've overlooked? any rb managemnet tool you particularly like? if you have any ideas please submit them here as comments or better yet via the UI preview. we'd really appreciate it. thanks.

for more information on the project see the "BoardFusion News Page" and the Project Wiki.

May 10, 2006

norwegian locale A-Go-Go

some recent work has me again turning over the rocks where core java locales are hiding and once again a closer look at what crawled out reveals just how sweet icu4j's locale support really is. according to several resources, such as ethnologue and the odin archive (gotta love that name), norway has two main written languages Bokmål and Nynorsk, with Bokmål being dominant. in core java there is one (well two if you include the plain norwegian langauge, no) locale and one variant for norway: no_NO and no_NO_NY. assumming core java meant Bokmål for plain Norwegian (no and no_NO), then i suppose the variant (no_NO_NY) is for Nynorsk. huh? but i thought Nynorsk was a language? why is it a variant here? in icu4j, which uses the CLDR for it's locale data, we can see two locales (four if you count the plain nb/Bokmål and nn/Nynorsk languages): nb_NO (Bokmål Norwegian) and nn_NO (Nynorsk Norwegian). neat and tidy.

one of the side effects of this core java locale is that ColdFusion's old locale name Norwegian (Nynorsk) actually produces no_NO locale data. any legacy apps still using this locale identifier are probably telling people the wrong thing, for example:
setLocale('Norwegian (Bokmal)');
writeoutput('#lsDateFormat(now(),"DDDD")#');
produces: mandag

while
setLocale('Norwegian (Nynorsk)');
writeoutput('#lsDateFormat(now(),"DDDD")#');
also produces: mandag


icu4j on the otherhand produces:
måndag for nn_NO
mandag for nb_NO

it looks like ColdFusion got tripped up on the "variant instead of language" locale.

taking this a step further, doing a "FULL" date format shows up even larger differences between core java and icu4j:
core java
8. mai 2006 for no_NO
8. mai 2006 for no_NO_NY

icu4j
måndag 8. mai 2006 for nn_NO
mandag 8. mai 2006 for nb_NO

oops. to my way of thinking, a "FULL" date format should include the day name as well as the rest of the date (date in month, month and year). i really wish ColdFusion would use icu4j.

and the "A-Go-Go" reference? nothing to with g11n or ColdFusion, just been listening to a lot of Dengue Fever lately and that song has just stuck in my head ;-)