bugGNU Octave - Bugs: bug #45331, unpack hangs when output file...

 
 

bug #45331: unpack hangs when output file exists

Submitted by:  Muhali <muhali>
Submitted on:  Tue 16 Jun 2015 10:13:36 AM UTC  
 
Category: Octave FunctionSeverity: 3 - Normal
Priority: 5 - NormalItem Group: None
Status: FixedAssigned to: Philip Nienhuis <philipnienhuis>
Originator Name: Open/Closed: Closed
Release: 4.0.0Operating System: GNU/Linux

Add a New Comment(Rich Markup)
   

You are not logged in

Please log in, so followups can be emailed to you.

 

(Jump to the original submission Jump to the original submission)

Sun 05 Jul 2015 09:23:57 AM UTC, comment #15:

Thanks for the detailed explanation.

I'll shut up and stop wasting your time :-)

Philip Nienhuis <philipnienhuis>
Project MemberIn charge of this item.
Sat 04 Jul 2015 12:38:51 PM UTC, comment #14:

The second patch won't unpack files to the current directory if called like so

Also, with gunzip ('../foo.gz') I get the following error:

This is because without the default assignment in the prototype of the function, dir is no longer a variable if gunzip is called with a single argument. Instead, when you call unpack you are calling it with the output of the function dir() which is a structure.

Rik <rik5>
Project Administrator
Thu 02 Jul 2015 08:29:59 PM UTC, comment #13:

Sure, but a similar stanza is in unpack.m.

How about the attached changeset? (passes all tests)

(file #34365)

Philip Nienhuis <philipnienhuis>
Project MemberIn charge of this item.
Thu 02 Jul 2015 02:58:51 PM UTC, comment #12:

The stanzas in gunzip and bunzip are important because they ensure that the archives are unpacked in the directory where the archive resides, rather than always in to the current directory.

Rik <rik5>
Project Administrator
Thu 02 Jul 2015 06:33:34 AM UTC, comment #11:

@Rik:
Thanks for looking at it.

I see you didn't touch gunzip.m and bunzip.m, while both have redundant if not dead code. I had two stanzas in my patch for that.

Shall I push another patch fixing those things?

Philip Nienhuis <philipnienhuis>
Project MemberIn charge of this item.
Wed 01 Jul 2015 09:21:50 PM UTC, comment #10:

I checked in a change on stable that does most of what was asked for (http://hg.savannah.gnu.org/hgweb/octave/rev/2fc43288a6c4).

Unpack no longer hangs because the '-f' switch is supplied to both gzip and bzip (I figured it was better to be consistent even though bzip doesn't require it).

I couldn't use the '-k' switch to keep-around the archive because that is a new gzip addition and not all Linux distros ship with a new enough version. So that remains a difference between Octave and Matlab. We could open up a separate Matlab Compatibility report about that.

The calculation of the output dir changed from the original patch because I found it was putting all unpacked files in the current directory rather than the directory where the archive resides.

Rik <rik5>
Project Administrator
Sat 20 Jun 2015 09:12:53 PM UTC, comment #9:

Changeset attached.

@Rik:
Would you please review and if satisfied push to, well, stable?

I'm not extremely proud of it but it works OK now and tests pass:
- unpacked archives are kept (added "-k" flags)
- "movefile" errors are gone
- no more hangs when gunzip'ing existing files (due to added "-f" flag)
==> as far as bunzip2 goes, I do not know if a forced overwrite is the way to go. Currently bunzip2 complains if it finds an existing file, as it does in a shell, but it doesn't hang Octave.

I think (and I already thought so 5 months ago when I first patched unpack.m) that unpack.m is in need of an overhaul but I lack time for that. It could use more tests as well - AFAICS only gunzip is tested.

(file #34280)

Philip Nienhuis <philipnienhuis>
Project MemberIn charge of this item.
Sat 20 Jun 2015 05:57:15 PM UTC, comment #8:

In gunzip.m & bunzip2.m the second input arg "dir" is [] (empty) by default.
Then, both functions have a stanza like this:

...which doesn't do an awful lot of things if dir = [].

Next, in unpack.m there's a problem with strcmp-ing otherwise identical paths with and without a trailing filesep.

I'll try to make something out of this.

Philip Nienhuis <philipnienhuis>
Project MemberIn charge of this item.
Fri 19 Jun 2015 09:19:57 PM UTC, comment #7:

Indeed:

At least some clues....

Philip Nienhuis <philipnienhuis>
Project MemberIn charge of this item.
Fri 19 Jun 2015 08:40:17 PM UTC, comment #6:

Thanks Philip. So it seems that the first part is correct, and that Octave both creates the gzip archive and also retains the original file.

The 'gunzip' action seems to be rather screwed up.

It looks like unpack.m needs some attention.

Rik <rik5>
Project Administrator
Fri 19 Jun 2015 08:19:40 PM UTC, comment #5:

With r2015b prerelease:

...and ML's file browser shows files "foo.mat" and "foo.mat.gz".

Now,

makes "foo.mat" reappear and "foo.mat.gz" is not deleted.

bzip2/bunzip2 isn't supported by Matlab on Windows and I haven't had time to install r2015b prerelease on Linux yet.

Philip Nienhuis <philipnienhuis>
Project MemberIn charge of this item.
Fri 19 Jun 2015 07:08:55 PM UTC, comment #4:

Can someone test the what Matlab does for

Does the original file foo remain in the directory? The ordinary Linux command gzip removes the original file, but I think we were trying for Matlab compatibility in this regard.

Rik <rik5>
Project Administrator
Wed 17 Jun 2015 06:15:43 AM UTC, comment #3:

@Philip: gzip has -f flag - force overwrite of output file and compress links.

Adding -f to gzip command doesn't hang Octave, but causes an error since there is command to move the file to itself.

Avinoam Kalma <avinoam>
Project Member
Tue 16 Jun 2015 06:47:48 PM UTC, comment #2:

same with bzip2. zip is different as it requires two arguments. The tar.* cases don't seem to hang.

Muhali <muhali>
Tue 16 Jun 2015 05:29:10 PM UTC, comment #1:

Does that happen with gzipped files only? How about bzip / zip / tar.bz / tar.gz / tgz ?

If gzip / gunzip has a sort of "force" or "overwrite" flag the first issue should be an easy fix.

As for the second, I think when g(un)zipping a file in the shell, the original file is deleted - at least that is how gzip/gunzip seems to work on my Linux systems.
However, in gzip.m's help text it is clearly stated that the original files will be kept.
Such a thing isn't mentioned in the gunzip.m help text, however.

Philip Nienhuis <philipnienhuis>
Project MemberIn charge of this item.
Tue 16 Jun 2015 10:13:36 AM UTC, original submission:

unpack hangs if the output file exists.

Only after pressing another return I get

error: unpack: unarchiving program exited with status: 2
gzip: /tmp/oct-qxMHrG/foo already exists; do you wish to overwrite (y or n)? not overwritten

Similarly for gunzip, ...

Moreover, when gzipping a file, the file itself is not deleted, whereas when gunzipping a file, that file is deleted.

Muhali <muhali>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach File(s):
   
   
Comment:
   

Attached Files
file #34365:  bug_45331_part2.m added by philipnienhuis (2KiB - text/x-objective-c)
file #34280:  bug45331.cs added by philipnienhuis (3KiB - text/x-csharp)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -unavailable- added by rik5 (Posted a comment)
  • -unavailable- added by avinoam (Posted a comment)
  • -unavailable- added by pantxo (Updated the item)
  • -unavailable- added by muhali (Submitted the item)
  •  

    Do you think this task is very important?
    If so, you can click here to add your encouragement to it.
    This task has 0 encouragements so far.

    Only project members can vote.

     

    Please enter the title of George Orwell's famous dystopian book (it's a date):

     

     

    Follow 11 latest changes.

    Date Changed By Updated Field Previous Value => Replaced By
    Thu 02 Jul 2015 08:29:59 PM UTCphilipnienhuisAttached File-=>Added bug_45331_part2.m, #34365
    Wed 01 Jul 2015 09:21:50 PM UTCrik5CategoryNone=>Octave Function
      StatusPatch Submitted=>Fixed
      Open/ClosedOpen=>Closed
      Releasedev=>4.0.0
    Sat 20 Jun 2015 09:12:53 PM UTCphilipnienhuisAttached File-=>Added bug45331.cs, #34280
      StatusIn Progress=>Patch Submitted
    Sat 20 Jun 2015 05:57:15 PM UTCphilipnienhuisStatusConfirmed=>In Progress
      Assigned toNone=>philipnienhuis
    Tue 16 Jun 2015 12:10:34 PM UTCpantxoStatusNone=>Confirmed
      Summaryunpack hangs file exists=>unpack hangs when output file exists

    Back to the top


    Powered by Savane 3.1-cleanup1