Lotus Notes is an old piece of software. It was inspired by PLATO Notes and DECNotes, and its development began in 1984. The first release was in 1989, though that’s not particularly old by IBM standards – IBM is still developing IMS, the database designed in 1966 as ICS/DL/I to keep track of the approximately three million parts that made up an Apollo Saturn V.
Nevertheless, Notes predated Java, JavaScript, Unicode, TCP/IP support in Windows, and many other things we now take for granted. Sometimes the age of Notes shows through in how the product works internally.
An example of this is the date and time handling. Originally Notes used basically the same time zone lists as Windows 1.0, which in turn was based on how MS-DOS did things. Dates and times were “automatically” converted to your time zone, and that was it. Back in the late 1990s it became clear that this was insufficient, so Notes was given a new type of field which could be used for selecting and storing time zone information. Internally, the time zone was stored as a set of undocumented keys and values packed into a string, for example:
Z=-1$DO=1$DL=3 -1 1 10 -1 1$ZX=130$ZN=W. Europe
Clearly Z is the Zulu offset, the number of hours from UTC. DO is Daylight Observed. DL is the rules for DST observance, ZX is likely the internal code number of the time zone in the old list, and ZN is the name of the time zone.
Unfortunately, time zones change quite regularly, because governments like to mess with them. It wasn’t long until changes were needed to the Notes and Domino time zone list. Unfortunately, the values in the string were stored in millions of user documents on customer servers around the world, so there was no way to easily rename (say) Tonga to Tongatapu. Nor was there any easy way to remove obsolete values. So instead, an extra layer of translation was added. The document might say ZN=Tonga in the time zone field, but that would be displayed as Nuku’alofa in the front end, that being the largest city on Tongatapu.
In the 1990s, Java was the new hotness. Early versions of Java had a terrible API for date and time handling; simple problems like producing a timestamp in Internet format could take ten lines of code. Nevertheless,
IBM decided to bet big on Java for enterprise software development. Notes and Domino were rebuilt, practically from the ground up, using Java EE and Eclipse technologies. Amazingly, full backward compatibility was kept with the old Windows version of Notes. This meant that when you started being able to develop Domino applications in Java, you found that as well as the Java Calendar and Date data types, there was also a DateTime class representing dates and times as handled by Notes since the 1980s.
Over the years, enough people complained about Java’s date and time APIs that it became clear that they needed replacing. Finally, Java 8 introduced a whole new date and time API.
If you haven’t learned Java 8 java.time APIs, well, they are fantastic compared to the old ones. Many problems which used to require error-prone hacks, such as how to represent a date with no time, now have straightforward and robust solutions.
Unfortunately, because it took until Java 8, there’s a huge amount of code out there which uses the old APIs. Oracle published some notes on how to convert between the old and new classes, but basically left the problem as an exercise for the programmer.
So it was that I decided to sit down and work out a utility class with all the conversions, and a set of unit tests to verify them. I called the utility HodgePodge, partly after the state of most Java date/time code, and partly after the hodge/podge transformer of Principia Discordia. I hoped that my HodgePodge class would help me reduce the general amount of chaos in my code.
Next, I turned my eye towards doing the same thing for IBM Notes and Domino DateTime values and time zone fields – that is, building utility functions to convert them to and from the new Java time API, since modern Domino development is done in Java and JavaScript.
This proved to be a tougher task. The first step was to build a list of Notes/Domino time zones, and work out how to convert them to Java 8 time zones, which are based on the Olson time zone database. To do this, I had to consult a number of sources.
First of all, I assembled a test database with every possible Notes/Domino time zone value represented, so I could examine both the front-end and back-end representations of them. I could then cross-reference that with Microsoft’s time zone list for Windows, and a somewhat out-of-date IBM list of Notes 4 time zones that someone helpfully included in an article about decoding hexadecimal universally unique IDs.
With all that information, I could often find an obvious semantic match in the Java 8 timezone list. If I couldn’t, my next step was to consult the site zeitverschiebung.net, which seems to catalog every single time zone for every major city with typical German thoroughness, in each case providing the Olson time zone name – which is usually (but not always) found in the Java 8 list.
For an example of an easy one, consider ZN=Eastern Standard Time (Mexico). There’s no Java/Olson entry under Mexico for any kind of Mexican Eastern Standard Time, but the Notes UI displays “Chetumal” in the time zone selector. Chetumal isn’t in Olson either, but if I look up Chetumal on the handy German web site, it tells me that the appropriate time zone is Mexico/Cancun. I know Cancun is a resort on the east coast of Mexico, which explains why it was called “Eastern Standard Time (Mexico)”, so that’s that one sorted.
It wasn’t long before I found some traps. How about ZN=US Mountain? That’ll be US Mountain Time, such as is observed in Colorado, so America/Denver, right? Wrong! The user interface control labels it Arizona. Arizona doesn’t observe DST, unlike Colorado, so the correct Java zone name is America/Arizona. There’s a separate ZN=Mountain for US Mountain Time.
Then there was ZN=Central America. That appears as Central America in the UI as well. It’s a Microsoft time zone, their page tells us that it represents “Central America Standard Time”, whatever that is, but no examples are given of anywhere which uses that time zone. However, looking at the underlying data structure it says DO=0, which means it doesn’t have DST. I went on vacation in Costa Rica and remembered that they don’t observe DST, because they’re a very sensible country. The offset matches, so America/Costa_Rica will do for that.
Then there’s TZ=US Eastern. You might think that that’s US Eastern Time (EST/EDT), but you’d be wrong because the value displayed to the end-user is Indiana (East). Indiana has been observing DST since 2006, so right now eastern Indiana is on EST/EDT – but there’s no guarantee that that will continue, so the best coding is probably America/Indiana/Indianapolis.
This sounds tedious, right? So you might be wondering if I could just use the Z value (offset from UTC) and DO value (DST observed) to sort things out. The answer, sadly, is no. It turns out that there’s both a Central Europe Daylight Time and a Central European Daylight Time, both abbreviated to CEDT, and they’re semantically different, but they currently both have the same UTC offsets and both observe DST. So if you want to convert times and dates with best fidelity, you can’t just use the UTC offset (with or without DST), and you can’t rely on the abbreviation for the time zone.
There’s a bigger problem, though – when the Domino DateTime object was created, Windows only supported integer time zone offsets – that is, whole numbers of hours. So the call to get the offset from UTC still only returns an integer. That’s a problem if you live in South Australia, where the time zone is +10:30. (And yes, quite a lot of people do live in Adelaide.)
So, what to do if the code needs a zoned time and date, but there’s no date/time field with a semantic value, just the offset found in the DateTime object? I could build an OffsetDateTime in Java, but there’s no way to get the necessary information. Or is there?
After thinking about the problem for a bit, I came up with a solution. The DateTime class has a method to return the date and time as a string, in its original internal time zone. It also has a method to return the date and time as a string in GMT. I could grab both, subtract one from the other, and that would tell me what the internal time zone offset was!
Unfortunately, the string format used by the DateTime object depends on the local i18n settings on the system where the code is running. So first, I had to write code which took the Notes/Domino i18n settings – provided as an International object – interrogated them, and built a format string to parse the appropriate date and time format using Java’s new DateTimeFormatter. I could then parse both my timestamps, calculate the time zone offset including the minutes, append that to the original timestamp, and parse that as an OffsetDateTime!
Oh, but it’s not that easy either. There’s a bug in Java 8 where it can’t parse timestamps with time zone offsets for which the number of hours is a single digit. It’s fixed in Java 9, but there’s no Java 8 backport, and Java 9 isn’t LTS so I’d have to wait for Java 11. So instead, I had to assemble my own ZoneOffset object, and combine it with a LocalDateTime to produce an OffsetDateTime.
Anyway, back to time zones. Australia was easy compared to some of the other values. Consider the time zone abbreviated as YW2.
The YW2 time zone is stored in Notes as ZN=Mid-Atlantic. The front end display is “Mid-Atlantic - Old”, which gives a clue as to the problem: no country actually uses that time zone any more. It used to be the time zone of Fernando de Noronha, an archipelago of 21 islands off the coast of Brazil with a population of around 2,800 inhabitants. However, Brazil’s eastern islands no longer observe daylight saving time (or horário de verão as they call it); only the west side of Brazil bothers with that.
So the Olson and Java time zone databases have no entry for “GMT-2:00 with DST”, hence nothing which corresponds to YW2. Which is a shame, as someone has managed to enter data in YW2 into one of the databases I manage, as I discovered while testing my code.
For that case, I decided to code YW2 as Etc/GMT+2 and accept that the data may be off by an hour for half the year, since I think the data is likely bogus to start with given the time zone attached. (Yes, GMT+2, because the Etc zones have + and - reversed to match POSIX and Microsoft conventions where positive time zones are west of Greenwich.)
Then there are a few values which are simply wrong in the current IBM Domino time zone list. The list has Samoa as UTC+13, but Samoa changed to UTC-11 (same as Niue) in 2011, so for a while it wasn’t in the Samoa Time Zone (defined by the US). Checking online suggests that Pago Pago (American Samoa) also moved time zone later on, so both Western Samoa and American Samoa are now UTC-11. Still, should I translate the time zone to American Samoa, Samoa (the nation), or even Pacific/Pago_Pago? I decided that since Samoa is an actual nation, references to Samoa without additional qualification should be taken to mean the nation, and I put Pacific/Samoa in the conversion function.
Finally, I went back and tested that all my test data produced valid semantic time zones when passed through the conversion functions, and results that were what I expected plus or minus an hour (because of DST being observed or not).
Overall, it was a gruelling and incredibly annoying experience, but I now have my web service providing events feeds in JSON with useful time zone information. To save any IBM customer from having to go through the same pain, I’m putting the code up on Github. Please let me know if you find any bugs!