bugGNU Octave - Bugs: bug #36677, canonicalize_file_name:...

 
 

bug #36677: canonicalize_file_name: inconsistent behavior for non-existing files

Submitted by:  Philip Nienhuis <philipnienhuis>
Submitted on:  Mon 18 Jun 2012 05:27:39 PM UTC  
 
Category: LibrariesSeverity: 3 - Normal
Priority: 5 - NormalItem Group: Incorrect Result
Status: PostponedAssigned to: None
Originator Name: Philip NienhuisOpen/Closed: Open
Release: devOperating System: Any

Add a New Comment (Rich MarkupRich Markup):
   

You are not logged in

Please log in, so followups can be emailed to you.

 

(Jump to the original submission Jump to the original submission)

Sun 20 Jan 2013 08:11:31 PM UTC, comment #12:

With some workarounds (if-else constructs) the io pkg scripts work OK.
Fixing this "bug" (rather: inconsistency) can wait as far as I'm concerned and as long no no one else reports to be affected.

Philip Nienhuis <philipnienhuis>
Project Member
Sun 02 Sep 2012 08:36:56 PM UTC, comment #11:

Thanks.

My C++ proficiency is simply too meager to be able to fix this.
But I can outline what the code might need to do:

The idea is of course to temporarily switch to the other drive to avoid do_make_absolute cluttering up.

Other than that on windows it doesn't work across drive letters (with files that exist or not), make_absolute_filename indeed fits the bill.

Philip Nienhuis <philipnienhuis>
Project Member
Sun 02 Sep 2012 04:57:18 PM UTC, comment #10:

make_absolute_filename() is defined in libinterp/interpfcn/utils.cc. However, I had already traced back the series of internal function calls to the function which needs work. That function is do_make_absolute at line 326 of liboctave/system/oct-env.cc.

Separately, I just updated the source tree for liboctave to group related function into subdirectories. You will need a version of the development sources that is greater than or equal to 15273:535bb601e7db.

Rik <rik5>
Project Administrator
Sun 02 Sep 2012 02:38:03 PM UTC, comment #9:

I (that is, "search") searched the entire octave source tree for "make_absolute_filename" but couldn't find it, except as a call in help.m.
In oct-env.cc I see do_make_absolute() and do_absolute_pathname().

Where is make_absolute_filename() defined?

Philip Nienhuis <philipnienhuis>
Project Member
Thu 30 Aug 2012 11:25:21 PM UTC, comment #8:

canonicalize_file_name() is never going to be the right function for this situation. It isn't particularly broken, it is just defined a particular way by GNU for their version of libc. Lots of people rely on the current behavior so there isn't much hope of changing it.

make_absolute_pathname() isn't a GNU or gnulib function but an Octave one. Therefore, we could potentially modify it to do whatever we want. The function is pretty short. If you are familiar with C++ you should take a look. The file is liboctave/oct-env.cc. The function is make_absolute() at line 326. I see that there is already a FIXME comment that the behavior may not be correct on all systems. Duh! It seems like this must have been coded in a bit of a hurry.

Rik <rik5>
Project Administrator
Thu 30 Aug 2012 10:49:53 PM UTC, comment #7:

"...begins with a drive letter..." = probably same as ispc().
Anyway that wouldn't make much difference as the second & third examples in the 1st post show. It would only solve part of the issue.

The issue is that a general mechanism is needed that also returns the full path of target dirs for non-existent files. Say I'm in /home/philip and I want to create a file in /tmp. Entering "../../tmp/newfile.fil" should return /tmp/newfile.fil.

For the same token I could say that the GNU behavior of canonicalize_file_name is broken.

Anyway some behavior on Windows. With 3.6.2 MSVC2010:
(we're in X:\ = root of drive X:)

so make_absolute_filename does return info (but faulty).
But now in subdir x:\octave\octave-3.7.x\bin:

...so there's a bug too in the sense that make_absolute_filename doesn't note that the ..\..\.. series descends below the "root level". Canonicalize_file_name seems robust (but one could argue that it incorrectly returns correct or rather expected info).

I'd better sort this out in more detail.....
Are there any tests for canonicalize_file_name and make_absolute_filename in the sources? Do they cover this?

And regarding gnulib:
what is the gnulib-defined behavior of make_absolute_filename?

Philip Nienhuis <philipnienhuis>
Project Member
Thu 30 Aug 2012 10:07:17 PM UTC, comment #6:

There was a bit of a miscommunication. Octave-Forge scripts shouldn't have to do anything with gnulib. Octave core needs to implement canonicalize_file_name consistently and it should be doing this by pulling the gnulib module. For some reason ./configure on your system didn't pull the module as it was supposed to. This is really a problem with the build system problem for core Octave.

But, if we can get gnulib working then canonicalize_file_name will definitely return an empty string. Thus, the original problem remains.

Maybe you can check the file name to see if it begins with a drive letter and if it doesn't you can use make_absolute_pathname.

Rik <rik5>
Project Administrator
Thu 30 Aug 2012 09:48:13 PM UTC, comment #5:

gnulib? AFAIK, until now gnulib is a bit alien to Octave-Forge.
Furthermore the spreadsheet functions are pure m-files. There's good reason to keep it that way.

The behavior you describe (GNU describes) simply kills the functionality that the spreadsheet functions need. Hopefully make_absolute_filename() will keep that functionality.

But I'll also investigate if full pathnames are really needed. If not this whole problem can simply be avoided.

Philip Nienhuis <philipnienhuis>
Project Member
Thu 30 Aug 2012 09:25:14 PM UTC, comment #4:

I suppose you should work around this for now. The working behavior of canonicalize_file_name is defined here (http://www.gnu.org/software/libc/manual/html_node/Symbolic-Links.html). It is very clear that the function should return an empty string if the file name object does not exist. We should probably me moving in the direction of getting the platforms to do the same thing. In this case, I don't exactly know why you aren't pulling the gnulib module.

Rik <rik5>
Project Administrator
Thu 30 Aug 2012 07:05:44 PM UTC, comment #3:

Rik, any plans to get this fixed soon? (I suppose you lack time)

If not I'll add a workaround for this to various functions in the OF io package and change status of this bug to "postponed".

Philip Nienhuis <philipnienhuis>
Project Member
Fri 29 Jun 2012 09:19:38 PM UTC, comment #2:

Thanks Rik.

Well, it works to some extent, but now I get other bugs (specific for Windows).
See below for a CLI session, sorry *nix users for the Windows-specific idiosyncrasies:

(Probably unneeded explanation for Windows haters:)
On windows you have a current directory on every drive (-letter). So while you're on drive C:, accessing a file on another drive depends on that other drive's current dir.
That's where make_absolute_filename() fails, but canonicalize_file_name() works correctly.

So yes, in principle if on *nix I could use make_absolute_filename() and on Windows canonicalize_file_name()

But, could canonicalize_file_name() be made to fall back on make_absolute_filename() on *nix, and vice versa on Windows?

That would be a "better" solution, not so much because I wouldn't have to fix the io pkg scripts then, but rather because the two functions would work consistently (cross-platform) in core Octave.

Philip Nienhuis <philipnienhuis>
Project Member
Fri 29 Jun 2012 07:49:20 PM UTC, comment #1:

Philip,

You have indeed found an inconsistency. Check the source code for file-ops.cc in the liboctave directory. The code has a number of #idefs. We first try to use the gnulib module, then we try to use the RESOLVEPATH module, then we look to see whether were on a Win32 platform.

For your specific application, would make_absolute_filename() work? It appears to work correctly for a non-existent file in the pwd, in a relative directory location '../', and in an absolute location '/tmp/'. The testing was done on a Linux platform.

Rik <rik5>
Project Administrator
Mon 18 Jun 2012 05:27:39 PM UTC, original submission:

canonicalize_file_name behaves differently under Linux than under Windows (file 'notfound.fil' doesn't exist):

Windows (both MSVC & MinGW) Octave-3.6.2_MSVC

Linux: (Octave-3.6.1 (Mandriva) & tip d.d. June 17, 2012)

Another example (on Linux:)

Some more background:

I wouldn't be able to tell for sure on which of the two platforms canonicalize_file_name behaves faulty (plus, I can't test on Mac OSX).
But the behavior is inconsistent across platforms.

As for me, I would prefer that on Linux, like on Windows, canonicalize_file_name assumes that a non-existing file would reside in the current directory.

This is a.o., needed in the UNO (OpenOffice.org) sections of the io package, where to be able to open or create a file, OOo needs the fully qualified file name.
On Windows the same seems to be needed for COM/ActiveX.

A clumsy workaround would be to first test for existence, and if nonexistent, prepend the filename with
['pwd' filesep]
but this would break for nonexistent files with fully qualified pathnames in another directory, as the third example shows.
Obviously, more involved workarounds can be imagined; but IMO that functionality should rather be contained in the canonicalize_file_name function.

Philip Nienhuis <philipnienhuis>
Project Member

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach File(s):
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -unavailable- added by mtmiller (Updated the item)
  • -unavailable- added by rik5 (Posted a comment)
  • -unavailable- added by philipnienhuis (Submitted the item)
  •  

    Do you think this task is very important?
    If so, you can click here to add your encouragement to it.
    This task has 0 encouragements so far.

    Only project members can vote.

     

    Please enter the title of George Orwell's famous dystopian book (it's a date):

     

     

    Follow 3 latest changes.

    Date Changed By Updated Field Previous Value => Replaced By
    Sun 19 Jan 2014 08:43:28 PM UTCmtmillerCategoryNone=>Libraries
    Sun 20 Jan 2013 08:11:31 PM UTCphilipnienhuisStatusNeed Info=>Postponed
    Fri 29 Jun 2012 07:49:20 PM UTCrik5StatusNone=>Need Info

    Back to the top


    Powered by Savane 3.1-cleanup