Solution For Rsync and Cygwin Daylight Savings Timezone Problem

["Wayne Piekarski" <wayne at cs dot unisa dot edu dot au>]

Hi,

I have been having a number of problems dealing with time zones and
synchronising files using Rsync to my linux machine on Cygwin with Windows
XP. I have read discussions previously on these mailing lists on how there
are problems that occur when there is a change in daylight savings time.

I saw a page by http://optics.ph.unimelb.edu.au/help/rsync/rsync_pc1.html
that discussed how to set the timezone to UTC and then do an ls to see if
the file dates are really the same. If there is a DST problem the suggestion
was to set --modify-window=3601 to set a 1 hour fuzzy factor to prevent the
problem. I had the same problem but didn't want to set this flag since I was
worried I would miss important updates on some files that had changed
shortly after an rsync run.

I did a lot of reading around about this and in this email I have developed
a solution to the problem that I have not seen anywhere else. My solution is
for WinXP with FAT filesystem but I am not sure what the effect is with NTFS
disks. From what I read up, FAT stores its dates in local time and not in
GMT. So it uses an offset from the clock settings to calculate the GMT value
to report to Rsync. The problem is that these DST calculations have a bug in
that if your clock moves over a DST boundary then files created on the
"other side" of the bounday will mysteriously get a 1 hour offset applied.
Unix avoids this problem by having proper DST calculations but Windows
doesn't do this right. The references below contain some discussions about
how these are calculated.

In Adelaide where I live, we left Daylight Savings Time at the end of March,
and so now my files are all 3600 seconds off relative to my Linux box. The
files were synchronised while DST is in effect. Now DST is off I need to do
something about this, I don't really want to resync 20 Gb of data over SSH.
If you run a 'stat' program over the files in question, the seconds since
1970 value will be off by 3600. If you change the time zone nothing seems to
change and the 1970 values on the Windows box remain the same. FAT is
supposed to use the time zone offset to calculate the GMT time but it
doesn't seem to take effect. You have to reboot the box! For some reason
parts of the TZ settings affect Windows immediately and others don't kick in
until many minutes later, so I just reboot to be safe - I spent half a day
to discover this, so it is important!

So all we need to do is get a time zone which is one hour away from where we
are now and it should apply a 1 hour offset to all the GMT values and then
this will fix our problem. I live in Adelaide which is GMT+9:30 normally, so
I need GMT+10:30. There is no time zone like this so we need to hack the
time zone files that Windows uses instead. I downloaded a tool from
Microsoft called TZEDIT.EXE that comes with some versions of Windows
resource kit. Using this tool, you can edit the time zone settings for the
current area you are in. I created a new time zone based on my previous
Adelaide one called "Cygwin Hacked" with GMT+10:30 and daylight savings
disabled.

Now after doing this you *MUST* activate the time zone change properly. Make
the change using TZEDIT, then you need to go to the control panel time/date
tool and click on the time zone and hit ok to make it take effect. You can't
enable the time zone and then edit it, you have to do it in the right order.
Then you must reboot the machine to make it take effect. (It may have a
delay to take effect but I don't know how long it is and I'm too lazy to
wait all day)

Now after the box reboots you will be in GMT+10:30 and when you run Rsync
everything just works like a dream, just like it did a week ago before DST
finished. You do not need to use --modify-window which could potentially
introduce errors and everything just works very well now.

In hindsight this problem could have been simplified if I had done the first
rsync when DST was not on. So if during July (I am in the southern
hemisphere) I did the rsync, then I could have just unticked the "support
DST" box in the clock control panel and could have avoided these problems.
Unfortunately because my backup was done when DST was enabled I have to now
create a time zone that simulates DST. For now this is my fix for the
problem but I am not sure if there are any other effects that I am not aware
of. Everything seems to be working fine.


Here are some references that I collected while researching this problem.
They contain a lot of information about the ways that Windows deals with
timestamps on NTFS and FAT filesystems.

http://optics.ph.unimelb.edu.au/help/rsync/rsync_pc1.html
http://list-archive.xemacs.org/xemacs-nt/199911/msg00130.html
http://sources.redhat.com/ml/cygwin/2002-12/msg01065.html
http://p2p.wrox.com/archive/c_plus_plus_programming/2001-06/53.asp
http://www.codeproject.com/datetime/dstbugs.asp
http://lists.samba.org/pipermail/rsync/2000-July/002491.html
http://support.microsoft.com/default.aspx?scid=kb;[LN];158588
http://www.cygwin.com/ml/cygwin/2000-06/msg00810.html



Here is the source code for a short program I wrote to find out what the
modified time of a file is, it is important that we print out the seconds
since 1970 GMT because this is the value that Rsync uses to determine if it
needs to copy the file over. I use this tool because it shows the seconds
values and is invariant to the use of the TZ environment variable and other
problems that can crop up.

#include <stdio.h>
#include <sys/stat.h>
#include <unistd.h>

void main (int argc, char *argv[])
{
  struct stat buf;

  if (argc != 2)
    exit (1);
  stat (argv[1], &buf);

  printf ("file %s\n", argv[1]);
  printf ("atime = %d  %s", buf.st_atime, ctime (&buf.st_atime));
  printf ("mtime = %d  %s", buf.st_mtime, ctime (&buf.st_mtime));
  printf ("ctime = %d  %s", buf.st_ctime, ctime (&buf.st_ctime));
}



Sorry for writing such a long email. This has been bugging me for ages and
each time DST kicks in it throws my backup system into chaos. I found pieces
of information all over the web but nothing really proposed a proper
solution to the problem so I wanted to put this all in one place so people
can find it in the future.

regards,
Wayne

----------------------------------------------------------------------------
Wayne Piekarski - PhD Researcher / Lecturer      pho:        +61-8-8302-3669
                                                 fax:        +61-8-8302-3381
Tinmith Project - Wearable Computer Lab          mob:           0407-395-889
Advanced Computing Research Centre               ema:  wayne at cs dot unisa dot edu dot au
University of South Australia                    www: http://www.tinmith.net



--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Bug reporting:         http://cygwin.com/bugs.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/