bugGNU Octave - Bugs: bug #67729, save function in a script has...

 
 

bug #67729: save function in a script has permission denied error and orphaned .saving_in_progress files

Submitter:  None
Submitted:  Mon 24 Nov 2025 05:29:56 AM UTC
   
 
Category:  Octave Function Severity:  4 - Important
Priority:  5 - Normal Item Group:  None
Status:  In Progress Assigned to:  None
Originator Name:  save Originator Email:  -email is unavailable-
Open/Closed:  * Open Release:  * 10.3.0
Operating System:  * Microsoft Windows Fixed Release:  None
Planned Release:  11.1.0 (current default)
* Mandatory Fields

Post a Comment

Add a New Comment Rich Markup
   

Discussion

Jump to the original submission

Mon 08 Dec 2025 05:34:38 PM UTC, comment #23: 

See the code I posted in comment #16.  The function _wrename is Windows-specific and there is a link to their documentation.  We seem to be using it correctly.  I'd prefer to use sync rather than any delay because there is unlikely to be a magic value that works on all computers.

Rik <rik5>
Group administrator
Mon 08 Dec 2025 05:18:11 PM UTC, comment #22: 

https://savannah. ... /?63803#comment25

See also above comment by JWE. Evidently Octave calls something called "_wrename" on Windows and that is outside gnulib?

Arun Giridhar <arungiridhar>
Group Member
Mon 08 Dec 2025 05:10:47 PM UTC, comment #21: 

If some of those errors are hsaring violations between Octave and windows services (indexer, AV, file browser), may be a retry loop on error?

Dmitri.
--

Dmitri A. Sergatskov <dasergatskov>
Mon 08 Dec 2025 05:06:41 PM UTC, comment #20: 

Octave calls unlink to delete the existing file before doing a rename operation.  I don't know what that is mapping to on Windows, but maybe we should add something like a call to sync after the operation to guarantee it has taken place.

Rik <rik5>
Group administrator
Mon 08 Dec 2025 05:04:53 PM UTC, comment #19: 

https://savannah. ... /?63803#comment38

Possibly related: the above comment by Markus about Linux vs Windows differences, which might explain why it fails on Windows in cases where it would work in Linux.

If it is indeed a race condition between closing the file and it being available for being renamed, would it help to add a delay of maybe 500 milliseconds or something after the file is closed on Windows? It doesn't solve the underlying cause but it would mitigate the effect on Octave saving files.

Arun Giridhar <arungiridhar>
Group Member
Mon 08 Dec 2025 04:08:44 PM UTC, comment #18: 

I found
https://stackover ... ssible-on-windows

In particular:
"Starting with Windows 10 1607, NTFS does support an atomic superseding rename operation."

Dmitri.
--

Dmitri A. Sergatskov <dasergatskov>
Mon 08 Dec 2025 03:48:52 PM UTC, comment #17: 

<<< Do you ever get the error message "Target already exists"?  Or is it always something else?  Or are there various error messages seen?

>>>


Kind of. See comment 12.

Also with:

$ cat /var/run/media/dmitri/DIESEL/t3.m
# Single element cell
c1 = {rand(60)};
for i = 1:20
  save(sprintf('test_c1_%d.dat', i), 'c1', '-binary');
endfor
disp('c1 done')

# Multiple element cell
c16 = cell(16,1);
for ix = 1:16
  c16{ix} = rand(60);
endfor
for i = 1:20
  save(sprintf('test_c16_%d.dat', i), 'c16', '-binary');
endfor
disp('c16 done')

$ cat /var/run/media/dmitri/DIESEL/t4.m
# Test struct
s.a = rand(60);
s.b = rand(60);
for i = 1:20
  save(sprintf('test_struct_%d.dat', i), 's', '-binary');
endfor
disp('struct done')



>> t3
c1 done
error: save: unable to save to test_c16_1.dat  File exists
error: called from
    t3 at line 14 column 3
>> t3
c1 done
c16 done
>> t3
error: save: unable to save to test_c1_8.dat  File exists
error: called from
    t3 at line 4 column 3
>> t4
struct done
>> t4
error: save: unable to save to test_struct_7.dat  File exists
error: called from
    t4 at line 5 column 3
>> t4
struct done
>> t4
error: save: unable to save to test_struct_19.dat  File exists
error: called from
    t4 at line 5 column 3


Sometimes I get an alternating "unable to open output file" and
"unable to save to xxx001  File exists"

I disabled AV (Windows defender), did not make a difference.

Dmitri.
--

Dmitri A. Sergatskov <dasergatskov>
Mon 08 Dec 2025 03:24:52 PM UTC, comment #16: 

@Markus: Adding you to the CC list for this bug since you have more experience with Windows than I do.

1) When Octave C++ code calls close() on an Octave file handle, does it map to C++ stdlib close()?  Or do we have a special #ifdef for Windows and we are using Windows native functions?

The base code is in load-save.cc at line 1608:

std::ofstream file = sys::ofstream (fname.c_str (), mode);

if (! file)
  err_file_open ("save", fname);

bool write_header_info = ! file.tellp ();

save_vars (argv, i, argc, file, format, save_as_floats,
           write_header_info);

file.close ();

This is followed by a call to rename at line 1630

std::string msg;
if (octave::sys::rename (fname, desiredname, msg) < 0)
  error ("save: unable to save to %s  %s",
         desiredname.c_str (), msg.c_str ());

One of the calls, either the close() or the rename(), seems to have problems because rename() fails.

If I look at the code for sys::ofstream in liboctave/system/oct-sysdep.cc I do see a slight difference for Windows code, but this seems to be about how the filename is encoded which shouldn't interfere with the library function.

std::ofstream
ofstream (const std::string& filename, const std::ios::openmode mode)
{
#if defined (OCTAVE_USE_WINDOWS_API)

  std::wstring wfilename = u8_to_wstring (filename);

  return std::ofstream (wfilename.c_str (), mode);

#else
  return std::ofstream (filename.c_str (), mode);
#endif
}

The file.close() call definitely uses standard library.

The problem might be in the rename function liboctave/system/file-ops.cc

int
rename (const std::string& from, const std::string& to,
        std::string& msg)
{
  int status = -1;

  msg = "";

  // Do nothing if source and target are the same file.
  if (same_file (to, from))
    return 0;

  // The behavior of std::rename with existing target is not defined by the
  // standard.  Implementations differ vastly.  For Octave, use the following
  // for the case that the target already exists:
  // If the source and the target are regular files, overwrite the target.
  // In other cases, fail.
  if (file_exists (to))
    {
      if (file_exists (to, false) && file_exists (from, false))
        unlink (to);
      else
        {
          msg = "Target already exists.";
          return status;
        }
    }

#if defined (OCTAVE_USE_WINDOWS_API)
  std::wstring wfrom = u8_to_wstring (from);
  std::wstring wto = u8_to_wstring (to);
  status = _wrename (wfrom.c_str (), wto.c_str ());
#else
  status = std::rename (from.c_str (), to.c_str ());
#endif

  if (status < 0)
    msg = std::strerror (errno);

  return status;
}

because I do see a Windows-specific function being used.  The documentation for the _wrename is at https://learn.mic ... ame?view=msvc-170.

@Gordon, @Dmitri: Do you ever get the error message "Target already exists"?  Or is it always something else?  Or are there various error messages seen?


Rik <rik5>
Group administrator
Mon 08 Dec 2025 01:48:46 PM UTC, comment #15: 

Yes, it works.
I also tried saving struct:

s.a = rand(60);
s.b = rand(60);
for i = 1:20
  save(sprintf('test_struct_%d.dat', i), 's', '-binary');
endfor
disp('struct done')

and it fails sporadically,
but multiple arrays always fine:


v1 = rand(60); v2 = rand(60); v3 = rand(60);
for i = 1:20
  save(sprintf('test_multi_%d.dat', i), 'v1', 'v2', 'v3', '-binary');
endfor
disp('multi done')


I am still not abandoned the AV theory (m.b. there is something in structures header that keeps AV to lock the file longer?).

In any case we need Windows people involve here.

Dmitri.
--

Dmitri A. Sergatskov <dasergatskov>
Mon 08 Dec 2025 01:35:07 PM UTC, comment #14: 

If it about rename not working on Windows with an existing file and a cell does this fail every time?

system ('touch tst.dat');
C = {randi([0, 1], 60)};
save -binary tst.dat C


Rik <rik5>
Group administrator
Mon 08 Dec 2025 01:02:21 PM UTC, comment #13: 

@Rik

All you simple tests pass for me w/o problem.

So saving "cell" seems critical.

Dmitri.
--

Dmitri A. Sergatskov <dasergatskov>
Mon 08 Dec 2025 12:52:09 PM UTC, comment #12: 

I ran the attached `t2.m` scripts. I saw:

>> t2
save error: xxx001
  Message: save: unable to save to xxx001  File exists
  Temp file exists after failure
>> t2
WARNING: xxx001.saving_in_progress already exists before save!
>> t2
>> t2
save error: xxx002
  Message: save: unable to save to xxx002  File exists
  Temp file exists after failure
>> t2
WARNING: xxx002.saving_in_progress already exists before save!
>> t2
save error: xxx002
  Message: save: unable to save to xxx002  File exists
  Temp file exists after failure
>> t2
WARNING: xxx002.saving_in_progress already exists before save!
>> t2
save error: xxx001
  Message: save: unable to save to xxx001  File exists
  Temp file exists after failure
>>


I think what is happening is that on Windows `rename` fails when destination exist, but on unix it just replaces it. So, my proposal:


diff -r af7dcb406dea libinterp/corefcn/load-save.cc
--- a/libinterp/corefcn/load-save.cc    Sun Dec 07 20:28:55 2025 +0100
+++ b/libinterp/corefcn/load-save.cc    Mon Dec 08 07:50:55 2025 -0500
@@ -1543,6 +1543,11 @@
       std::string desiredname = sys::file_ops::tilde_expand (argv[i]);
       std::string fname = desiredname + (append ? "" : ".saving_in_progress");

+      // Remove any stale .saving_in_progress file from a previous
+      // failed or interrupted save.
+      if (! append)
+        sys::unlink (fname);
+
       i++;

       // Matlab v7 files are always compressed
@@ -1627,6 +1632,10 @@

       if (! append)
         {
+          // On Windows, rename() fails if the destination file exists.
+          // Remove the destination file first to ensure rename succeeds.
+          sys::unlink (desiredname);
+
           std::string msg;
           if (octave::sys::rename (fname, desiredname, msg) < 0)
             error ("save: unable to save to %s  %s",


I cannot build for Windows, so cannot really test. Also if we do
make some changes -- could we apply that to `stable` so people can check nightly builds?

(file t2.m)

Dmitri.
--


(file #57930)

Dmitri A. Sergatskov <dasergatskov>
Mon 08 Dec 2025 12:36:20 PM UTC, comment #11: 

@Dmitri: Excellent that it fails repeatedly.  I'd still like to find the MWE.  Going back to my original example, does this fail?

N = 10;

x = rand (1e6,1);

for i = 1:N
  save (sprintf ('xsave_%d.dat', i), 'x', '-binary');
endfor



Rik <rik5>
Group administrator
Mon 08 Dec 2025 11:12:46 AM UTC, comment #10: 

I changed save format to `-binary` and immideately got 10 save errors (on Win10). So we are getting somewhere...

Dmitri.
--

Dmitri A. Sergatskov <dasergatskov>
Mon 08 Dec 2025 10:35:32 AM UTC, comment #9: 

@Gordon

Do you have any kind-of anti-virus software on your computer (besides usual Windows Defender, or whatever it is called nowadays)?

I tried your script on old Win 10 laptop (Core2 Duo cpu) running it off USB2 stick (i.e. your script was on the stick and it saved the files on this stick). I cannot get any failures.

I also tried running this on linux with thread sanitizer and it does not find any issues with possible issues in CLI mode.

Dmitri.
--


Dmitri A. Sergatskov <dasergatskov>
Mon 08 Dec 2025 09:16:47 AM UTC, comment #8: 

I agree with point 1: This is not an issue on Linux and is restricted to MS Windows.

Question 1: Did the tests using '-text' versus '-binary' show any difference?

I'd really like to fix this for the upcoming Octave 11 release because it should never be the case that your data is accidentally, and occasionally, lost when saving.


Rik <rik5>
Group administrator
Fri 05 Dec 2025 03:34:10 PM UTC, comment #7: 

Here's a summary of my testing with observations and opinions:

(1) I've now used my test script (with 10 iterations, data size of 115kb) on a Linux system (AMD Ryzen 7) and had no "save" errors. My opinion: this error only occurs in a Windows environment. I will do no further testing on Linux systems.

(2) The system which most reliably produces the error is a Windows system on a laptop (slower CPU?, slower I/O?). I've tested under Octave's GUI and CLI. The rename error occurs under both environments but less frequently under CLI. I've tested with the "Synchronize box un-checked" and it still occurs but even less frequently under GUI and now only on the first "save" request.

(3) The data size and type is irrelevant. The number of iterations of a test is actually counter-productive. If I do a test with 10 iterations and repeat it quickly in a row I get no errors. If I do a test and wait (I/O routines paged out?) I can induce the error, usually on the first of 10 tries. Sometimes it occurs on test 1 and 10. Sometimes on test 1, 4, 7, and 9. Sometimes not at all. After all, it is timing dependent on what happens AFTER the data is written and BEFORE the rename operation is done. The file must be free and available for the rename operation to succeed. This may require a kludgy solution in Octave's "save" routine: create a semaphore when finished writing to a file and don't do the rename until the semaphore is cleared.

Conclusion: After all this, I'm just using my try/save/catch/rename workaround for now. If anyone has any other ideas, I'll be glad to test them.

Gordon Chamberlain <inprosys>
Thu 04 Dec 2025 04:15:53 PM UTC, comment #6: 

@Gordon:

Are you running it in GUI or console? I wonder if this is a thread contention issue between gui file browser and the main octave thread. If so, you can try to un-check `Synchronize ...` box in GUI preferences --> File Browwser

Dmitri.
--

Dmitri A. Sergatskov <dasergatskov>
Thu 04 Dec 2025 04:00:32 PM UTC, comment #5: 

You can see where I was going.  In order to debug this issue it would be really helpful to have a script that repeatably fails.  And it would be useful at the end of the process for verifying that any fix applied actually makes a difference.

I guess we can try your SYS-3 system since a 40% failure rate is high enough to be noticed.

Test 1

Change the save command to

save (fileName, 'modelCell', '-ascii');

If you don't specify a format then it will use the output of default_save_options() which might be binary or text.  Since my test script worked, I want to clarify whether '-ascii' is relevant.

Test 2

Change the save command to

save (fileName, 'modelCell', '-binary');

Same reasoning as above.  Maybe  '-binary' works.


Rik <rik5>
Group administrator
Wed 03 Dec 2025 07:29:47 PM UTC, comment #4: 

First of all, thanks to Rik for suggesting a test script to re-create the problem. However, since it seems to be a timing problem, it's going to be elusive to re-create.

I have three test systems on which to test this problem -- all Windows 11. -- SYS-1 has Octave 10.2.0 and a Ryzen 9 
-- SYS-2 has Octave 10.3.0 and an Intel i7-4790K
-- SYS-3 has Octave 10.3.0 laptop with Intel Ultra 7 155H

Using Rik's test script, the problem never occurs -- on all three systems.

Here's my results on each system:

SYS-1
After running Rik's script, I executed a single save request with my data in an Octave Command Window and it failed.  Is it data size dependent?  Rik's example has a file size of around 20,000kb and my data was sized at around 115kb (a cell array of 16 elements each sized 60 x 60). 
I placed my save request (115kb) in a shorter loop of 10 iterations (with try/catch logic) and it ran without any failures. ???

SYS-2
All loop tests ran without failures.  However, when running a long script which calls other functions (that execute a save operation), it ALWAYS failed.

SYS-3
Rik's loop test ran without failure.  However, the loop using my data (with try/catch logic) had intermittent failures. Runs 1, 4, 5, and 7 had failures; runs 2, 3, 6, 8, 9, and 10 ran without any trouble. Really???

I don't know what else to try.  Any ideas?

My test script:
modelCell = cell(16,1);
for ix = 1:16
  modelCell(ix) = {randi([0,1],60)};  %% (60 x 60) of 0's and 1's
endfor
for i = 1:10
  fileName = ['xxx' (num2str(i,'%3.3i'))];
  try
    save (fileName, 'modelCell');    %% each file approx. 115kb
  catch
    disp(['save error: ' fileName])
  end_try_catch
endfor

Gordon Chamberlain <inprosys>
Wed 03 Dec 2025 11:39:34 AM UTC, comment #3: 

I can't replicate this, but I am using Linux.  This may be specific to MS Windows.  I'm attempting to find a minimum working example.  See code below and attached as tst_save.m

N = 100;

x = rand (1e6,1);

for i = 1:N
  save (sprintf ('xsave_%d.dat', i), 'x', '-ascii');
endfor

@Gordon: If you run this test script can you reliably reproduce the problem?


(file #57907)

Rik <rik5>
Group administrator
Wed 03 Dec 2025 05:59:01 AM UTC, comment #2: 

Very nice. The Cobalt Blue method works good for sparrows.

Instead of rename file can also be moved. On Linux they are same but on windows they are not. This works on windows server 2008 but that is old os so you must test on young windows.

Is octave 11 related to windows 11? Why not became 11 at same time(

Anonymous
Tue 02 Dec 2025 10:31:08 PM UTC, comment #1: 

I'm also having this error. Octave implements save in three steps: (1) create file with .saving_in_progress extension; (2) write data to file (3) rename file without any extension. If step (2) has not completed and the file not synchronously closed, step (3) fails because the file is not available for renaming - hence permission denied. Recommendation, step (2) must end with a synchronous close so that the rename in step (3) will work correctly.

Workaround code:
try
  save("fileName", "variable");
catch
  rename("fileName.saving_in_proress", "fileName");
end_try_catch

I've tested this workaround on several systems and it has always worked correctly.  Obviously, a correctly behaving Octave would be appreciated.

Gordon Chamberlain <inprosys>
Mon 24 Nov 2025 05:29:56 AM UTC, original submission:  

Saving a file will work through the command window with manually running commands line by line, but get a permission denied error when trying to run the script and the file is partially saved as a .saving_in_progress file.   Tried in several folders and manually changing write permissions and even running as adminsitrator.

save ABCD.txt a -ascii;

Windows 11 24H2


Anonymous

 

Attached Files

Attached Files
file #57930:  t2.m added by dasergatskov (704B - text/x-objcsrc)
file #57907:  tst_save.m added by rik5 (101B - text/x-octave)

(Note: upload size limit is set to 4.0MiB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

 

Dependencies

This item does not depend on any other items.

Digest:
   bug dependencies.

 

Mail Notification Carbon-Copy List

Carbon-Copy List
  • -email is unavailable- added by arungiridhar (Posted a comment)
  • -email is unavailable- added by rik5
  • -email is unavailable- added by dasergatskov (Posted a comment)
  • -email is unavailable- added by rik5 (Updated the item)
  • -email is unavailable- added by inprosys (Posted a comment)
  •  

    Votes

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only group members can vote.

     

    Please enter the title of George Orwell's famous dystopian book (it's a date):

    History

    Follow 8 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2025-12-08 rik5 Carbon-Copy- Added mmuetzel
    2025-12-08 dasergatskov Attached File- Added t2.m, #57930
    2025-12-08 rik5 Planned ReleaseNone 11.1.0 (current default)
    2025-12-08 rik5 StatusNeed Info In Progress
    2025-12-08 rik5 Severity3 - Normal 4 - Important
    2025-12-03 rik5 Attached File- Added tst_save.m, #57907
        StatusNone Need Info
    2025-11-26 mmuetzel Dependencies- bugs #67730 is dependent

    Back to the top

    Powered by Savane 3.16-a7ba.
    Corresponding source code