This is the mail archive of the cygwin-talk mailing list for the cygwin project.


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]
Other format: [Raw text]

Re: Mirroring and stage and tmp dirs


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Igor Peshansky wrote:
> On Sun, 16 Apr 2006, Robert Pendell wrote:
> 
>> Igor Peshansky wrote:
>>> On Sun, 16 Apr 2006, Robert Pendell wrote:
>>>
>>>> Should we be ignoring these folders during mirroring?  I was mirroring
>>>> them but recently started getting permission denied errors during
>>>> transfers for those folders so they are now excluded from rsync
>>>> transfers (i.e. not on my public mirror).  What is the stage folder used
>>>> for?
>>> The /stage folder is used to build up parts of the changed directory
>>> structure and transfer them over in one swell foop (to avoid a race
>>> with the upset cron job, where it could catch the uploader in
>>> mid-update and result in a broken setup.ini).
>>>
>>> In short, you should not mirror that directory, as nothing in there is
>>> going to be useful on a mirror.
>>>
>>>> rsync: send_files failed to open "/stage/tcp_wrappers-7.6-2.sh" (in
>>>> cygwin-ftp): Permission denied (13)
>>>> rsync: send_files failed to open "/stage/tcp_wrappers_7.6.tar.gz" (in
>>>> cygwin-ftp): Permission denied (13)
>>>>
>>>> And from earlier on...
>>>> rsync: send_files failed to open "/stage/monotone-0.26-1.sh" (in
>>>> cygwin-ftp): Permission denied (13)
> 
> Heh, I just looked closely at the names of the files...  Looks like
> someone unpacked a source archive there -- not part of your normal "create
> files/directories and move them to /release" sequence...  Don't know why
> this happened, but my explanation below is probably not correct for this
> particular case.
> 
>>> You probably caught the uploader in the act of moving files from
>>> /stage to /release.
>>> 	Igor
>> Ok.  I updated the cron job last night to exclude those folders from now
>> on.  I've been trying to optimized the command so it causes as few 404's
>> as possible.  It is setup to delete files after getting the new ones but
>> there are still a few occuring.
>>
>> Oh and an interesting statistic.  So far this month I have had more hits
>> from people using Wget then the regular setup program.  _Alot_ more hits.
>>
>> Wget (various versions) - 76841
>> Cygwin Setup - 26035
>>
>> This is versus last month where Cygwin and Wget was very close.  I
>> wonder if people are starting to branch (2nd tier) off my mirror...
> 
> I doubt it.  Can you also check whether the hits are for the current
> binary tarballs, or for the older source/binary ones?  If the former, it
> might indicate the more prominent usage of clean_setup.pl (which builds
> lists of wget commands).  If the latter, people may be doing what I
> occasionally do -- grabbing the whole older tarball to extract an older
> version of some file...  Also, are some tarballs hit more than others?
> 	Igor

Unfortunately the host is only providing Analog stats at the moment
which is limited on what information it provides and the raw logs are
only for the past week.

On the 11th I got a crap load of hits from a specific ip running Wget
1.10.2 and it was grabbing alot of directory listings.  It also grabed a
great deal of files too though.  In fact there were over 2000 files
retrieved via Wget from this one ip address.  It also went through the
stage, mail-archives, and tmp folders.  Alot of the directory listings
had "?N=D", "?M=A", "?S=A", "?D=A" appended to them.  What are these for?

- --
Robert Pendell
shinji257@uplink.net

Thawte Web of Trust Notary
CAcert Notary
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.2.2 (MingW32)

iD8DBQFEQoQ0NqnRaBCOmrYRAsDNAJ4k6LUiqOiR6/BMwmDTnNGFtCE9JACgsDjO
mKAopq0X3D68vH42Omgg/UE=
=Ygoa
-----END PGP SIGNATURE-----


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]