1

Resolved

Time zone remaining ToUniversalTime and ToLocalTime issues

description

Sorry about the delay in my response. I tested your last round of fixes and we are now much closer to having it working right.
 
Thanks for letting me know about DateTimeKind.Utc. I made DateTimeKind explicit in all my tests, for example:
 
zone.ToUniversalTime(New DateTime(DateTime.Parse("2008-03-30 00:00").Ticks, DateTimeKind.Local))
 
You said "Also, I don't think the ToLocalTime tests are really valid on the boundary cases." I know, you should never have a case of '2006-04-02 02:30' in America/Chicago time zone since 2 am becomes 3 am, creating an hour 'hole'. I think you are doing the right thing in this case by applying the DST rule and not worrying about this "invalid" hour:
 
zone.ToUniversalTime('2006-04-02 01:00'): '4/2/2006 7:00:00 AM'
zone.ToUniversalTime('2006-04-02 01:30'): '4/2/2006 7:30:00 AM'
zone.ToUniversalTime('2006-04-02 02:00'): '4/2/2006 7:00:00 AM'
zone.ToUniversalTime('2006-04-02 02:30'): '4/2/2006 7:30:00 AM'
 
The same (repeated hour) issue affects .ToLocalTime when DST goes off. Again, you are doing the right thing IMO.
 
The remaining issues revolve around the interpretation of the DST switchover time rule. The Olsen database uses several letters to indicate how the DST switch time should be interpreted. Here are a few examples:
 
---------------------------------------------------
DST rule name Valid on/off Date Time Offset 
---------------------------------------------------
No letter:
US >= 2007 on Mar Sun>=8 2:00 1:00 
'u':
EU on Mar lastSun 1:00u 1:00 
's':
Russia on Mar lastSun 2:00s 1:00 
 
The key to interpreting the letter is (see http://home.tiscali.nl/~t876506/TZworld.html):
 
no letter or w: wall clock time, actual local time.
s: local standard time (winter time)
u or g or z: UTC time.
So, if a 'DST on' time is given as '2:00s', this means a switch on 2:00 local time. If a 'DST off' time is given as '2:00s', this means a switch on 3:00 local time (assuming a DST offset of 1 hour).
 
In zone.ToUniversalTime and zone.ToLocalTime, the switchover time is not always interpreted correctly. Here are the use cases. Please see a complete listing of use case results and the source code in the attached file 'UTC Testing.rtf' to see the results for dates in different years.
 
  1. America/Chicago (rule for 2006: Apr Sun>=1 2:00 [local time]): ToUniversalTime switchover time is correct:
     
    zone.ToUniversalTime('2006-04-02 01:00'): '4/2/2006 7:00:00 AM'
    zone.ToUniversalTime('2006-04-02 01:30'): '4/2/2006 7:30:00 AM'
    zone.ToUniversalTime('2006-04-02 02:00'): '4/2/2006 7:00:00 AM' <- switchover is correct
    zone.ToUniversalTime('2006-04-02 02:30'): '4/2/2006 7:30:00 AM'
     
  2. America/Chicago (rule for 2006: Apr Sun>=1 2:00 [local time]): ToLocalTime interprets the switchover time as UTC (u) instead of local. The switch occurs at 2am UTC time (9 pm on the preceding day local time) instead of 8 am UTC time (2 am local):
     
    zone.ToLocalTime('2006-04-02 01:30'): '4/1/2006 7:30:00 PM'
    zone.ToLocalTime('2006-04-02 02:00'): '4/1/2006 9:00:00 PM' <- switchover occurs here (2:00 UTC)
    zone.ToLocalTime('2006-04-02 02:30'): '4/1/2006 9:30:00 PM'
    zone.ToLocalTime('2006-04-02 03:00'): '4/1/2006 10:00:00 PM'
    zone.ToLocalTime('2006-04-02 07:00'): '4/2/2006 2:00:00 AM' <- instead of here (2:00 local actual time)
    zone.ToLocalTime('2006-04-02 07:30'): '4/2/2006 2:30:00 AM'
     
  3. Europe/Paris (Mar lastSun 1:00u - 'u' means 'UTC'): ToUniversalTime acts as if the switchover time is expressed in local time:
     
    zone.ToUniversalTime('2007-03-25 00:00'): '3/24/2007 11:00:00 PM'
    zone.ToUniversalTime('2007-03-25 00:30'): '3/24/2007 11:30:00 PM'
    zone.ToUniversalTime('2007-03-25 01:00'): '3/24/2007 11:00:00 PM' <- switchover occurs here (1:00 local time)
    zone.ToUniversalTime('2007-03-25 01:30'): '3/24/2007 11:30:00 PM'
    zone.ToUniversalTime('2007-03-25 02:00'): '3/25/2007' <- and should occur of here (1:00 UTC)
     
  4. Europ/Paris (Mar lastSun 1:00u - 'u' means 'UTC'): ToLocalTime switchover time is correct:
     
    zone.ToLocalTime('2007-03-25 00:00'): '3/25/2007 1:00:00 AM'
    zone.ToLocalTime('2007-03-25 00:30'): '3/25/2007 1:30:00 AM'
    zone.ToLocalTime('2007-03-25 01:00'): '3/25/2007 3:00:00 AM' <- switchover is correct
    zone.ToLocalTime('2007-03-25 01:30'): '3/25/2007 3:30:00 AM'
     
  5. Europe/Moscow (Mar lastSun 2:00s = 's' means 'local standard time'): when standard time is switching to DST, .ToUniversalTime acts correctly. This is probably because at this point local actual time and local standard time are the same:
     
    zone.ToUniversalTime('2007-03-25 01:00'): '3/24/2007 10:00:00 PM'
    zone.ToUniversalTime('2007-03-25 01:30'): '3/24/2007 10:30:00 PM'
    zone.ToUniversalTime('2007-03-25 02:00'): '3/24/2007 10:00:00 PM' <- switchover is correct
    zone.ToUniversalTime('2007-03-25 02:30'): '3/24/2007 10:30:00 PM'
     
  6. Europe/Moscow (Mar lastSun 2:00s = 's' means 'local standard time'): when DST is switching to standard time, .ToUniversalTime acts as if the switchover time rule is expressed in local time and not in local standard time:
     
    zone.ToUniversalTime('2007-10-28 01:00'): '10/27/2007 9:00:00 PM'
    zone.ToUniversalTime('2007-10-28 01:30'): '10/27/2007 9:30:00 PM'
    zone.ToUniversalTime('2007-10-28 02:00'): '10/27/2007 11:00:00 PM' <- switchover occurs here (01:00 'standard time')
    zone.ToUniversalTime('2007-10-28 02:30'): '10/27/2007 11:30:00 PM'
    zone.ToUniversalTime('2007-10-28 03:00'): '10/28/2007' <- and should occur here (02:00 'standard time')
    zone.ToUniversalTime('2007-10-28 03:30'): '10/28/2007 12:30:00 AM'
     
  7. Europe/Moscow (Mar lastSun 2:00s = 's' means 'local standard time'): ToLocalTime acts as if the switchover time is expressed in UTC and not local standard time:
     
    zone.ToLocalTime('2007-03-24 23:00'): '3/25/2007 2:00:00 AM' <- switchover should occur here (2:00 'standard time')
    zone.ToLocalTime('2007-03-25 01:00'): '3/25/2007 4:00:00 AM'
    zone.ToLocalTime('2007-03-25 01:30'): '3/25/2007 4:30:00 AM'
    zone.ToLocalTime('2007-03-25 02:00'): '3/25/2007 6:00:00 AM' <- instead of here (2:00 UTC)
    zone.ToLocalTime('2007-03-25 02:30'): '3/25/2007 6:30:00 AM'
     
    Example 2:
     
    zone.ToLocalTime('2007-10-27 23:00'): '10/28/2007 3:00:00 AM' <- switchover should occur here (2:00 'standard time' or 3:00 local actual time)
    zone.ToLocalTime('2007-10-28 01:00'): '10/28/2007 5:00:00 AM'
    zone.ToLocalTime('2007-10-28 01:30'): '10/28/2007 5:30:00 AM'
    zone.ToLocalTime('2007-10-28 02:00'): '10/28/2007 5:00:00 AM' <- instead of here (2:00 UTC)
    zone.ToLocalTime('2007-10-28 02:30'): '10/28/2007 5:30:00 AM'
     
  8. Related issue: zone.GetDaylightChanges doesn't appear to indicate whether the switchover datetime is UTC, local current time, or local standard time. It might be useful to accept a parameter in this method that specifies which of the 3 times one wants returned. Or, constrain it to UTC? Personally, I don't need this, but it will probably come up as an issue for someone:
     
    zone.GetDaylightChanges(2007).Start: '3/11/2007 2:00:00 AM'
     
    Thanks in advance.

file attachments

comments

schizoidboy wrote Sep 8, 2007 at 4:12 AM

Again, great work creating very structured use cases. It helps a lot to create regression tests.

I remembered parsing out the modifiers (u, s, w, etc.) from the tz database, but I never actually used them :-). Shows how much I know. The data is there so I just have to add the proper logic. Should have this in 0.2.16.0 soon...

As for GetDaylightChanges, I hadn't spent much time on the semantics. It is part of the System.TimeZone contract. I remember reading discussion on MS forums that this was a poorly designed method (it was actually a concession by an MS employee) due to lack of semantics just as you point out. TzTimeZone currently returns the UTC DateTime with kind of Local, if I remember correctly, so that is clearly a problem. I can't redefine the semantics of the overriden GetDaylightChanges, but I can add an overload which takes a DateTimeKind... I will look into it...

Thanks again! I hope to have the fixes soon...

schizoidboy wrote Sep 8, 2007 at 9:32 PM

Hi marioigrec,

So, I've found the problems with assuming that the date switches were in UTC when they were actually in local time; however, in your use case, I'm a bit confused by:

zone.ToLocalTime('2006-04-02 07:00'): '4/2/2006 2:00:00 AM' <- instead of here (2:00 local actual time)

Doesn't this go back to the issue of the ambiguity of UTC without context. In this case, we are passing a UTC time of 2006-04-02 07:00, and you expect that this is 2:00 local time, but actually it could either be 2:00 local time or 3:00 local time, right?

I would prefer in this case not to add the save time.

I'm still working on the broader changes of adding what I call the "inflectionKind." The changes are much broader than I expected, so this might take some time, and there might be some regressions in other functionality as well which I need to validate...

Thanks!

schizoidboy wrote Sep 8, 2007 at 11:49 PM

This has hopefully been fixed in release 0.2.16.0 (http://www.codeplex.com/publicdomain/Release/ProjectReleases.aspx?ReleaseId=6962). Please verify and there is also a question above about UTC semantics.

I've also added a new method "public virtual DaylightTime GetDaylightChanges(int year, DateTimeKind inflectionKind)" which accepts the DateTime Kind the daylight changes should be returned in.

Thanks!

MARIOIGREC wrote Sep 21, 2007 at 3:56 PM

I will test .16 version shortly. Thank you so much for all these fixes. Re this:

"Doesn't this go back to the issue of the ambiguity of UTC without context. In this case, we are passing a UTC time of 2006-04-02 07:00, and you expect that this is 2:00 local time, but actually it could either be 2:00 local time or 3:00 local time, right?"

Well, the resolution is ambiguous when you are converting the opposite way, but what you are doing in this case is correct IMO. This is the moment in time when the rule makes the 2 am hour repeat, so we're good. I will retest taking this into account and let you know.

MARIOIGREC wrote Sep 21, 2007 at 7:50 PM

OK, found just one case that I think is off by one hour. Here's the output of my tests with annotations:

DST off rule for Europe/Moscow: Oct lastSun 2:00s (3am (23:00 UTC) becomes 2am)zone.ToLocalTime('2007-10-27 22:00'): '10/28/2007 2:00:00 AM'zone.ToLocalTime('2007-10-27 22:30'): '10/28/2007 2:30:00 AM'zone.ToLocalTime('2007-10-27 23:00'): '10/28/2007 3:00:00 AM' <- I think this should be 2:00 am...zone.ToLocalTime('2007-10-27 23:30'): '10/28/2007 3:30:00 AM' <- ...and this should be 2:30 am...zone.ToLocalTime('2007-10-28 00:00'): '10/28/2007 3:00:00 AM' <- ...instead of 4 am becoming 3 amzone.ToLocalTime('2007-10-28 00:30'): '10/28/2007 3:30:00 AM'zone.ToLocalTime('2007-10-28 01:00'): '10/28/2007 4:00:00 AM'

If you agree with my interpretation of the rule "2:00s", which is that 3am (23:00 UTC) should become 2am, then I think we should should see 23:00 UTC converted to 2:00 instead of 3:00. In that case my tests would look like this:

zone.ToLocalTime('2007-10-27 22:00'): '10/28/2007 2:00:00 AM'zone.ToLocalTime('2007-10-27 22:30'): '10/28/2007 2:30:00 AM'zone.ToLocalTime('2007-10-27 23:00'): '10/28/2007 2:00:00 AM'zone.ToLocalTime('2007-10-27 23:30'): '10/28/2007 2:30:00 AM'zone.ToLocalTime('2007-10-28 00:00'): '10/28/2007 3:00:00 AM'

I wanted to confirm that this behavior is not different in different zones with an "s" DST rule, so I tested Asia/Novosibirsk:

DST off rule for Asia/Novosibirsk: Oct lastSun 2:00s (3am (20:00 UTC) becomes 2am)zone.ToLocalTime('2007-10-27 19:00'): '10/28/2007 2:00:00 AM'zone.ToLocalTime('2007-10-27 19:30'): '10/28/2007 2:30:00 AM'zone.ToLocalTime('2007-10-27 20:00'): '10/28/2007 3:00:00 AM'zone.ToLocalTime('2007-10-27 20:30'): '10/28/2007 3:30:00 AM'zone.ToLocalTime('2007-10-28 21:00'): '10/29/2007 3:00:00 AM'zone.ToLocalTime('2007-10-28 21:30'): '10/29/2007 3:30:00 AM'zone.ToLocalTime('2007-10-27 22:00'): '10/28/2007 4:00:00 AM'

and the behavior is consistent.

This is an insignificant issue for my needs, but may cause problems for someone automating a global manufacturing system or managing a large distribution system.

And I apologize if my terminology (3am "becomes" 2am) is confusing - I mean 3am "should be interpreted as" 2am.

MARIOIGREC wrote Sep 21, 2007 at 7:56 PM

Additional tests (Asia/Novosibirsk) are attached as UTC_Testing2.txt (http://www.codeplex.com/publicdomain/WorkItem/AttachmentDownload.ashx?WorkItemId=12480&FileAttachmentId=5042).