bugGNU Octave - Bugs: bug #62365, history fails to create nested...

 
 

bug #62365: history fails to create nested directory on write attempt

Submitter:  Jim Maloney <jmaloney65>
Submitted:  Mon 25 Apr 2022 03:36:25 PM UTC
 
Category:  Interpreter Severity:  3 - Normal
Priority:  5 - Normal Item Group:  Unexpected Error or Warning
Status:  Fixed Assigned to:  None
Originator Name:  Jim Maloney Open/Closed:  * Closed
Release:  * dev Operating System:  * Any
Fixed Release:  None Planned Release:  None
* Mandatory Fields

Add a New Comment (Rich Markup)
   

Comment Type & Canned Response:
       

 

( Jump to the original submission )

Fri 21 Oct 2022 06:17:23 AM UTC, comment #18: 

Creating the (nested) folders for the history file seems to have worked on all GitHub action runners.
Closing as fixed.

Markus Mützel <mmuetzel>
Project Member
Thu 20 Oct 2022 06:08:25 PM UTC, comment #17: 

I pushed the patch (minus two typos) here:
https://hg.savannah.gnu.org/hgweb/octave/rev/6a5e4ef80a95

And followed up with adding a test whether this is working on Linux, macOS and Windows:
https://hg.savannah.gnu.org/hgweb/octave/rev/1d99e68c05c0

Marking as ready for test.

Markus Mützel <mmuetzel>
Project Member
Sat 15 Oct 2022 03:48:07 PM UTC, comment #16: 

The attached patch adds a new function that creates a directory making sure that all parent directories are also created.
I chose the name `recursive_mkdir` because we already have `recursive_rmdir` for a similar purpose. But the function doesn't actually call itself recursively.
The patch also uses that new function when creating the directory for the history file. That will hopefully fix the issue that users (especially on Mac platforms) are seeing when multiple levels of the directory for the history file don't exist.

Are there testers on macOS that could try this patch?

Also moving to "dev" because it might be too risky to change this for the final release of Octave 7, and a work-around exists.

(file #53861)

Markus Mützel <mmuetzel>
Project Member
Wed 27 Apr 2022 06:16:09 PM UTC, comment #15: 

Fwiw, the lines where `mkdir` is called:
https://hg.savannah.gnu.org/hgweb/octave/file/ba701853b1bf/liboctave/util/cmd-hist.cc#l373

            // Try to create the folder if it does not exist
            std::string hist_dir = sys::file_ops::dirname (f);
            if (! hist_dir.empty ())
              {
                sys::file_stat fs (hist_dir);
                if (! fs.is_dir () && (sys::mkdir (hist_dir, 0777) < 0))
                  (*current_liboctave_error_handler)
                    ("%s: Could not create directory \"%s\" for history",
                     "gnu_history::do_write", hist_dir.c_str ());
              }

Markus Mützel <mmuetzel>
Project Member
Wed 27 Apr 2022 05:28:03 PM UTC, comment #14: 

> Does octave expect that .local/share does always exist on unix-like systems?


Looking at the documentation of `history_file`, I think it does.

Markus Mützel <mmuetzel>
Project Member
Wed 27 Apr 2022 05:23:19 PM UTC, comment #13: 

Confirmed. The function uses POSIX `mkdir` [1] which doesn't create nested directories IIUC.
Afaict, there is no POSIX function that would create nested directories. We'd need to spin one ourselves.
Rather than doing that, we could maybe use `std::filesystem::create_directories` which is part of the filesystem library standardized in C++17 [2]. We already started talking about whether it's time to allow C++17 code in Octave. But we didn't come to a conclusion yet...

[1]: https://pubs.opengroup.org/onlinepubs/009695299/functions/mkdir.html
[2]: https://en.cppreference.com/w/cpp/filesystem/create_directory

Markus Mützel <mmuetzel>
Project Member
Wed 27 Apr 2022 05:21:42 PM UTC, comment #12: 

Then it should be possible to create the whole folder hierarchie with

make -p

Does octave expect that .local/share does always exist on unix-like systems?

Torsten Lilge <ttl>
Project Member
Wed 27 Apr 2022 05:06:21 PM UTC, comment #11: 

Interestingly, I was able to manually create these directories

jim.maloney@JGMs-MacBook-Pro sphere24 % mkdir /Users/jim.maloney/.local            
jim.maloney@JGMs-MacBook-Pro sphere24 % mkdir /Users/jim.maloney/.local/share
jim.maloney@JGMs-MacBook-Pro sphere24 % mkdir /Users/jim.maloney/.local/share/octave

perhaps the software is trying to create the tree all at once

now history -w does not fail

now history is being saved and loaded per session

Jim Maloney <jmaloney65>
Wed 27 Apr 2022 05:01:34 PM UTC, comment #10: 

Is it expected that users cannot create directories in their home directory on macOS? Is there something wrong with that path that prevents it from being created on macOS? What would be a "good" fallback to use on macOS instead?

Markus Mützel <mmuetzel>
Project Member
Wed 27 Apr 2022 04:58:15 PM UTC, comment #9: 

octave:1> history -w
error: gnu_history::do_write: Could not create directory "/Users/jim.maloney/.local/share/octave" for history

Yes, it throws an error

Jim Maloney <jmaloney65>
Wed 27 Apr 2022 04:54:48 PM UTC, comment #8: 

From the documentation of history_file:

> The default value is $DATA/octave/history, where $DATA is the platform-specific location for (roaming) user data files (e.g., $XDG_DATA_HOME or, if that is not set, ~/.local/share on Unix-like operating systems or %APPDATA% on Windows). The default value may be overridden by the environment variable OCTAVE_HISTFILE.


It looks like MacOS isn't following the XDG Base Directory Specification [1]. And the fallback directory is used.

Does it print any errors if you execute `history -w`?

[1]: https://specifications.freedesktop.org/basedir-spec/basedir-spec-latest.html

Markus Mützel <mmuetzel>
Project Member
Wed 27 Apr 2022 04:36:33 PM UTC, comment #7: 

octave:1> history_file
ans = /Users/jim.maloney/.local/share/octave/history

yes, not a valid path

cd /Users/jim.maloney/.local            
cd: no such file or directory: /Users/jim.maloney/.local

Jim Maloney <jmaloney65>
Wed 27 Apr 2022 04:34:18 PM UTC, comment #6: 

What does `history_file` return? Does it return a valid path?

Markus Mützel <mmuetzel>
Project Member
Wed 27 Apr 2022 04:30:39 PM UTC, comment #5: 

You are correct - the command-history is coming in empty

running octave 7.1.0 on my. mac, executing history shows just the history line

after a few other commands, history then shows them and the up-arrow works

so, yes the previous command history is either not getting saved or not getting loaded properly

Jim

Jim Maloney <jmaloney65>
Tue 26 Apr 2022 02:10:04 PM UTC, comment #4: 

After starting a new Octave session, what does the history command show?  If there are no entries in the list, then up arrow has nothing to move to.  Maybe somehow the history list is not preserved/restored from previous sessions?

John W. Eaton <jwe>
Project Administrator
Tue 26 Apr 2022 01:04:33 PM UTC, comment #3: 

I don't have access to a Mac. So, I can't test myself.

Possibly a mis-configured GNU Readline?

Markus Mützel <mmuetzel>
Project Member
Tue 26 Apr 2022 11:54:58 AM UTC, comment #2: 

Installed with Homebrew

Non-gun was a typo - running it in non-gui mode.  Just ascii in the terminal window

Strangely, after a keyboard command a the script, up-arrow will allow one to access the command history typed since the keyboard command.

Jim Maloney <jmaloney65>
Tue 26 Apr 2022 05:08:22 AM UTC, comment #1: 

How did you install Octave? Homebrew, macports, self compiled, ...?
Is this with the GUI or with the CLI?
What does "non-gun" mean?

Markus Mützel <mmuetzel>
Project Member
Mon 25 Apr 2022 03:36:25 PM UTC, original submission:  

Using 7.1.0 Octave on my mac, the up arrow no longer is showing the previous commands

Jim Maloney <jmaloney65>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by ttl (Posted a comment)
  • -email is unavailable- added by jwe (Posted a comment)
  • -email is unavailable- added by mmuetzel (Posted a comment)
  • -email is unavailable- added by jmaloney65 (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only project members can vote.

     

     

     

     

    Follow 12 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2022-10-21 mmuetzel StatusReady For Test Fixed
        Open/ClosedOpen Closed
    2022-10-20 mmuetzel StatusPatch Submitted Ready For Test
    2022-10-15 mmuetzel Attached File- Added bug62984-mkdir-recursive.patch, #53861
        StatusConfirmed Patch Submitted
        Release7.1.0 dev
    2022-04-27 mmuetzel Item GroupNone Unexpected Error or Warning
        StatusNeed Info Confirmed
        Operating SystemMac OS Any
        Summaryup arrow no longer shows previous command (Mac, CLI) history fails to create nested directory on write attempt
    2022-04-26 mmuetzel Summaryup arrow no longer shows previous command (Mac, non-gun) up arrow no longer shows previous command (Mac, CLI)
    2022-04-26 mmuetzel StatusNone Need Info

    Back to the top


    Powered by Savane 3.9