Wed 05 Jan 2011 06:18:16 AM UTC, original submission:
I apologize for not being able to give real info here. All the specifics are proprietary info belonging to my company.
I have 2 scripts. One calls another with different parameters, each is in the background. The second script calls wget on an ftp location to get the list of files on the site. It parses them and then downloads as appropriate.
Simplified example:
Each of these is being called by a separate instance of the secondary script running in the background.
wget --no-verbose 'ftp://foo.com/' -O '/downloads/foo/foo_listing.html' >foo.log
wget --no-verbose 'ftp://bar.com/' -O '/downloads/bar/bar_listing.html' >bar.log
When I run the secondary script once at a time, everything behaves as expected. I get an html file with a list of files, links to them, and information about the files the same way I would when viewing an ftp url through a browser.
Continued simplified one at a time (and expected) example results:
foo_listing.html:
...
<a href="ftp://foo.com/foo1.xml">foo1.xml</a> ...
<a href="ftp://foo.com/foo2.xml">foo2.xml</a> ...
...
bar_listing.html:
...
<a href="ftp://bar.com/bar3.xml">bar3.xml</a> ...
<a href="ftp://bar.com/bar4.xml">bar4.xml</a> ...
...
When I run the secondary script many times in the background, some of the resulting files, although they have the base urls correct (the one that was passed in) the files listed are from a different run of wget.
Continued simplified multiprocessing (and actual) example results:
foo_listing.html:
...
<a href="ftp://foo.com/bar3.xml">bar3.xml</a> ...
<a href="ftp://foo.com/bar4.xml">bar4.xml</a> ...
...
bar_listing.html
correct, as above
|