bugGNU Octave - Bugs: bug #51203, [octave forge] (io)...

 
 

bug #51203: [octave forge] (io) xlsopen/xlsclose may create spurious files/folders instead of .xlsx file with "OCT" interface

Submitter:  Andrey Aleshin Igorevich <iamandreyaleshin>
Submitted:  Thu 08 Jun 2017 07:12:03 AM UTC
   
 
Category:  Octave Package Severity:  2 - Minor
Priority:  3 - Low Item Group:  Incorrect Result
Status:  Fixed Assigned to:  philipnienhuis
Originator Name:  Open/Closed:  * Closed
Release:  * other Operating System:  * Microsoft Windows
Fixed Release:  None Planned Release:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Mon 25 Sep 2017 04:31:08 AM UTC, comment #82: 

I don't know what to say. Thank you both for solve this task. But me very pity that I didn't have a opportunity to realize all my brain.
Have a good day
Andrey

Andrey Aleshin Igorevich <iamandreyaleshin>
Sun 24 Sep 2017 06:47:13 PM UTC, comment #81: 

Markus,

I pushed your fix in comment #79 here:
http://hg.code.sf.net/p/octave/io/rev/66b5b97039f0
Thank you very much for your perseverance & patience.

Closing report

Philip Nienhuis <philipnienhuis>
Group Member
Sun 17 Sep 2017 07:27:16 PM UTC, comment #80: 

I'll have a look this week. Thank YOU for your patience and persistence.

Philip Nienhuis <philipnienhuis>
Group Member
Sun 17 Sep 2017 07:08:22 PM UTC, comment #79: 

Hi Philip,
The attached patch should be cleaner and also fix some typos in the previous one. Thanks for your patience.

(file #41820)

Markus Mützel <mmuetzel>
Group administrator
Tue 12 Sep 2017 08:13:20 PM UTC, comment #78: 

Thx Markus.
I'll await a new patch (no hurry, when you have time :-) )

Philip Nienhuis <philipnienhuis>
Group Member
Mon 11 Sep 2017 05:09:05 PM UTC, comment #77: 

Hi Philip,

No worries. Whenever you have time.

Below some comments to your questions:


__OCT_spsh_close__.m

    Where is variable adddToFile (L.23) used?


I forgot to delete this line, which was only useful during debugging.


    What is the use of hunk L.30-35?


If I recall correctly, InfoZip messed with relative paths if the zip-file that it creates was NOT located in the same directory as the source files. Those lines move the zipped file created in the temp directory (line 27) to the destination directory and rename it from .zip to .xlsx.


__OCT_spsh_open__.m
-------------------

    What is the use of variable i_open in L.76? It is nowhere used


Debug variable I forgot to delete for the final patch. The same with t_unpack in line 60. Sorry.


    Why do you want:

- fid = fopen (sprintf ('%s/xl/workbook.xml', tmpdir));

to be changed by

+ workbook_xml = sprintf ('%s/xl/workbook.xml', tmpdir);
+ fid = fopen (workbook_xml);


Same as above. Please ignore.

Looking further below in the patch (lines 92-97), it looks like I might have messed up the indentation (not sure if I read the patch correctly). You might want to ignore that hunk as well...

Sorry for the messy patch. I shouldn't try and hurry...
When I have some minutes, I'll clean the patch.

Markus Mützel <mmuetzel>
Group administrator
Sun 03 Sep 2017 12:57:59 PM UTC, comment #76: 

@Markus
sorry for long delay, after returning from vacation there were (and still are) other (private) distractions keeping me busy.

I went through your patch of comment #74 and have some Q. (line numbers are those of the patch):

_OCT_spsh_close_.m

  1. Where is variable adddToFile (L.23) used?


  1. What is the use of hunk L.30-35?


_OCT_spsh_open_.m
-------------------

  1. What is the use of variable i_open in L.76? It is nowhere used


  1. Why do you want:


-    fid = fopen (sprintf ('%s/xl/workbook.xml', tmpdir));

to be changed by

+    workbook_xml = sprintf ('%s/xl/workbook.xml', tmpdir);
+    fid = fopen (workbook_xml);

It merely introduces a temporary variable that is nowhere used anymore.

Philip Nienhuis <philipnienhuis>
Group Member
Thu 20 Jul 2017 09:16:37 AM UTC, comment #75: 

@Markus
Dear Markus, I want to offer some my thinks:
Do you think, may be the next title is more informative?
&quot;io's xlsopen/close with &quot;OCT&quot; interface are not stable in non empty workfolder (ssd only)&quot;?

Andrey Aleshin Igorevich <iamandreyaleshin>
Wed 19 Jul 2017 08:46:36 AM UTC, comment #74: 

Philip, thanks for pushing that patch, too.
I attached a patch with the remaining changes and changed the title accordingly. (Maybe you can come up with something a little less wordy?)
We can come back to this (rare) issue after you have returned if you like.
Enjoy your free days!

(file #41242)

Markus Mützel <mmuetzel>
Group administrator
Tue 18 Jul 2017 08:10:43 PM UTC, comment #73: 

@Markus,
I pushed the patch in comment #69 as well, thanks.

What shall we do with this bug report?
By the end of next week I'll be away for 3 weeks and until then I have little time left to devote to Octave.

  • If you think the unicode issues are solved we can at least adapt the title.


  • Corrupted .xlsx:

That may have been caused (in part or completely) by the (lack of) unicode conversion too, I hit that several times myself.

As to Infozip and "cd": I know the issue is real but rare, and chasing it down has shown to be quite hard.

In the original bug report Andrey states that it is the COM interface that gave him errors. probably that issue is not so much that the file is corrupt but rather that is has the wrong extension, or contents => Excel stubbornly creates .xlsx files even if the filename extension is .xls (an Excel bug if you ask me).
That reminds me of the fact that the default file extension currently still is .xls. But now that most people have MS-Office >= 2007 .xlsx is a better default. So I just changed that:
http://hg.code.sf.net/p/octave/io/rev/5b5b31fa4020



Philip Nienhuis <philipnienhuis>
Group Member
Mon 17 Jul 2017 08:10:07 AM UTC, comment #72: 

@Markus

No-No. It's my eng words.

Don't worry. Take a log.

No even 1 bug. And this is a clean result
This is my reply of your Saturday's letter. I'm write here to quickly answer. I said my mail works fine home only.

(file #41219)

Andrey Aleshin Igorevich <iamandreyaleshin>
Mon 17 Jul 2017 08:03:22 AM UTC, comment #71: 

@Andrey: I am not sure whether I understood correctly, sorry. Did the tests fail horribly? Or did it succeed without any error?
Did you try with the version without the loop around the "cd" command?

Markus Mützel <mmuetzel>
Group administrator
Mon 17 Jul 2017 04:40:30 AM UTC, comment #70: 

@Dear Markus:
2000 iterations w 0.4-0.5 sec per iteration - no even 1 - clean result. At my work with ssd.

Andrey Aleshin Igorevich <iamandreyaleshin>
Sun 16 Jul 2017 08:23:30 PM UTC, comment #69: 

Thank you, Philip, for pushing that patch.
I attached yet another one for a few changes in _OCT_spsh_open_.m and _OCT_spsh_close_.m that should be less controversial.

(file #41213)

Markus Mützel <mmuetzel>
Group administrator
Sun 16 Jul 2017 04:53:00 PM UTC, comment #68: 

@Markus:
I pushed your cset (comment #25) for using native2unicde rather than utf82unicode (and reverse functions) here:
http://hg.code.sf.net/p/octave/io/rev/dbd2adbb3ef1
with some minor style fixes (parens around if conditions).
Thanks for this patch.

Philip Nienhuis <philipnienhuis>
Group Member
Sat 15 Jul 2017 11:43:48 AM UTC, comment #67: 

.ods and .xlsx are zipped archives containing 2 or more trees of subdirs, all (including the "root" dir) containing XML files.

xlswrite (or rather some private functions that are called in a chain of other functions) unzips such an archive to a temporary subdirectory, modifies it, and in the end zips it back into the original place, overwriting the original .xlsx/.ods file.
After processing the temporary subdir with the unzipped archive is wiped.

xlsread and xlsfinfo (or rather private functions .....) merely do the same but don't modify it and in the end just wipe the temporary subdir.

The hairy issue here is that this bug is very hard to reproduce; sometimes it pops up, most of the time it doesn't. It makes one think of OS issues behind the scenes (like temporary swapping to disk of, or briefly blocking access to, required system libraries).
At one time I figured that maybe the original archive was somehow locked by zip/unzip, and releasing the lock might be delayed, but AFAIU that isn't the case.
But the thing that I suspect the most on at least Windows is virus/malware scanners. These are explicitly blocking all access to new files until those files can be declared "safe".  (Which doesn't explain why I saw this bug mostly on fast HDDs/SSDs.)

In the io package's spreadsheet test functions there are pause(0.25) statements after each call to xlswrite (or odswrite) to avoid any occurrence of this bug.
On my two dev boxes I just tried with all delays set to 0 (i.e., pause(0), or no delay) and I couldn't reproduce this bug at all anymore. Next week I can try on my PC at work where I hit this bug more often.

Philip Nienhuis <philipnienhuis>
Group Member
Fri 14 Jul 2017 08:58:49 PM UTC, comment #66: 

I tried using the script in comment #65 and didn't see a problem.

I'm using io 2.4.7 on a Windows system with a build of dev Octave (Octave hg id 5296657cde44) but I'm not sure what mxe-octave version.  It was built around the beginning of May.

My approach to debugging this problem would be to simplify the xlswrite function to the point where it just does the minimum to demonstrate the problem.

Exactly what is xlswrite doing in this case?

John W. Eaton <jwe>
Group administrator
Fri 14 Jul 2017 08:29:52 PM UTC, comment #65: 

Attached is a changed patch that doesn't use the loop around "cd".
Andrey used the following script to trigger the bug:

more off
cell1={'One, two, three, 4'};

filename = [mfilename('fullpath'), '.xlsx'];
for i= 1:300
  fprintf ("-------------- iteration %d ---------------\n", i);
  if exist(filename, "file")==2
    disp("deleting file")
    delete(filename);
  end
  if exist(filename, "file")==2
    warning ("File NOT deleted!");
  end
  t_xlsx = tic;
  xlswrite (filename, cell1, [], [], 'oct');
  toc (t_xlsx)
  if !exist (filename)
    warning ("xlsx file NOT created!");
  endif
endfor


Could you please first try with the original files in io 2.4.7 whether the script triggers the bug for you? Does the attached patch still fix the issue?


(file #41206)

Markus Mützel <mmuetzel>
Group administrator
Tue 11 Jul 2017 02:51:20 PM UTC, comment #64: 

Philip, can you increase "max_no_change" to 200 (or something higher) and try again. Maybe also increase t_max if this were necessary.
In the discussion in bug #51388 I understood that the very similar code there was triggering the bug for you. Maybe it needs more iterations to be triggered?

I will try and get the dependencies for some (or all) of the other interfaces within the next few days. Will report back when I have some results.

Markus Mützel <mmuetzel>
Group administrator
Mon 10 Jul 2017 04:56:20 PM UTC, comment #63: 

This is what I get, writing your script as 'test_cd_delay.m':

>> addpath (pwd)
>> somedirectory1 = 'c:\tmp'
somedirectory1 = c:\tmp
>> somedirectory2 = 'f:\tmp'
somedirectory2 = f:\tmp
>> test_cd_delay
Elapsed time is 0.0816691 seconds.
t_delay = 0
num_no_change =  7
>> test_cd_delay
Elapsed time is 0.086797 seconds.
t_delay = 0
num_no_change =  7
>> test_cd_delay
Elapsed time is 0.0738969 seconds.
t_delay = 0
num_no_change =  7


To run my suggestions in comment #61 you'd need at least Java (just a JRE), plus the required Java .jar libraries for each interface. LibreOffice is an extra. If you're on Windows and have Excel installed, you'd also need the Windows package loaded.

I have the following:

>> dir ([getenv("USERPROFILE") filesep "java"])
.                                 odfdom-java-0.8.8-incubating.jar
..                                poi-3.15.jar
OpenXLS-10.jar                    poi-ooxml-3.15.jar
commons-collections4-4.1.jar      poi-ooxml-schemas-3.15.jar
dom4j-1.6.1.jar                   xercesImpl-2.11.0.jar
gwt-servlet-deps.jar              xercesImpl.jar
jOpenDocument-1.4rc2.jar          xml-apis-2.11.0.jar
jxl.jar                           xmlbeans-2.6.0.jar


and LibreOffice (64-bit) is in 'C:\Program Files\LibreOffice 5' (needing the following .jars & other Java stuff), taken from chk_spreadsheet_support.m's listing:

>> chk_spreadsheet_support ('', 3)
:
:
UNO/Java (.ods, .xls, .xlsx, .sxc) <OpenOffice.org>:  - program OK
  - unoil.jar OK
  - jurt.jar OK
  - juh.jar OK
  - unoloader.jar OK
  - ridl.jar OK
  => UNO (OOo) OK


If you don't want to install all of that (which I'd understand), maybe we can wait for someone else who runs Windows and does have Java etc installed.

Philip Nienhuis <philipnienhuis>
Group Member
Mon 10 Jul 2017 03:38:28 PM UTC, comment #62: 

Philip,

Currently, I don't have LibreOffice or even Java installed. I always was using the OCT interface. If I understood correctly, both are needed to trial all available interfaces. Am I missing anything else?

Meanwhile, can you check the following code snippet for two directories that failed the test in bug #51388 (comment #2 there)?

t_delay = 0; % start test with no delay
t_inc = 0.1; % increment delay each time bug was triggered
max_no_change = 5; % number of successful (outer) loops before we trust the delay
t_max = 5; % approx. maximum time for test to run

num_no_change = 0;
t_start = tic;
while (num_no_change++ <= max_no_change) && (toc (t_start) < t_max)
  cd (somedirectory1)
  pause (t_delay);
  while ~strcmpi (somedirectory1, pwd)
    warning ('bug triggered');
    t_delay += t_inc;
    num_no_change = 0;
    cd (somedirectory1)
    pause (t_delay);
  endwhile
  cd (somedirectory2)
  pause (t_delay);
  while ~strcmpi (somedirectory2, pwd)
    warning ('bug triggered');
    t_delay += t_inc;
    num_no_change = 0;
    cd (somedirectory2)
    pause (t_delay);
  endwhile
endwhile

toc (t_start)
t_delay
num_no_change

if num_no_change <= max_no_change
  warning ("No reliable delay time found within t_max.")
endif


It runs quite fast on a machine that doesn't experience the error (50-60ms for me).
You might need to tweak the abort criteria (first 4 lines) to get reliable results...

Markus Mützel <mmuetzel>
Group administrator
Sun 09 Jul 2017 08:05:10 PM UTC, comment #61: 

Markus,

Only the OCT interface invokes "cd" explicitly.

If you look in the test functions
io_xls_testscript.m
io_ods_testscript.m
you'll note many pause statements for most if not all interfaces. All of those were needed to circumvent errors due to the OS apparently not being ready for the next command in those scripts.

Setting the value of "dly" (the delay time) to zero will help to illustrate where the errors exactly occur for which interface.

Results on your computer(s) would be interesting.
I was thinking that the "while strcmpi (pwd, ..)" loop could be morphed into a piece of code that assesses a machine-specific required delay time for each machine it runs on at the start of each first invocation of xlsopen or xlsclose, using persistent or global variables. That while loop may need to exit after the total accumulated delay would reach a max. of 1 second or so to avoid a hang.

Philip Nienhuis <philipnienhuis>
Group Member
Wed 05 Jul 2017 09:46:11 PM UTC, comment #60: 

@Markus

As to other affected interfaces, I'll sort that out (again).
A notable example is UNO (invoking LibreOffice behind the scenes using a Java-UNO bridge). LibreOffice invokes a built-in zip procedure to unzip/zip .ods and .xlsx files, and it was with UNO that I first hit this bug some years ago.


As to move to core:
I doubt if all of io (spreadsheet & XML) can be moved to core; I think only XML and the OCT spreadsheet interface will do.
The COM and Java-based interfaces could/should probably be left behind in a weeded-out io package. COM relies on Excel itself + the windows package so that is obviously out-of-the-question (although COM is a "system library" in GPL terms). Furthermore most octave devs don't care much for Java and if we include Java-based interfaces we'd introduce new dependencies.

AFAICS most of the required preparatory work has already been done, I need to polish and update it.

Philip Nienhuis <philipnienhuis>
Group Member
Wed 05 Jul 2017 05:09:55 PM UTC, comment #59: 

Philip,
can you give me a pointer to which other interfaces are affected? I could only find OCT using zip/unzip or unpack or manually changing directories.

I read through the threads you linked to. I would definitely like to see the io package merged into core.
If you think I can help getting the files ready for the move, please send a PM or an email with what you think should be done (or something more public if that is preferred). I will try and see if I can be of any support.

Markus Mützel <mmuetzel>
Group administrator
Wed 05 Jul 2017 12:40:39 PM UTC, comment #58: 

Markus,
No i'm not at all opposed to fixing io for the time being. But it'll have to be fixed for all affected interfaces, not just OCT.
Later this week I'll look into that.
FYI, fixing OF packages ahead of core releases while keeping backward compatibility is merely business-as-usual :-)

Some other background that may be relevant:
At OctConf 2017 I spoke with JWE about moving spreadsheet + XML I/O to core Octave (an old wish of him, see
http://octave.1599824.n4.nabble.com/Spreadsheet-support-in-core-Octave-instead-of-io-package-tt4666947.html#a4666955
and
https://lists.gnu.org/archive/html/octave-maintainers/2015-05/msg00074.html).
At OctConf 2017 I hoped it could happen around (last) June. Well it now looks like that'll be a bit later... :-)

One thing I've been preparing for a.o. that move to core, is eliminating duplicate code. We now have various xls....m functions doing similar things as ods....m counterparts; some time ago I've combined them and made the one just a wrapper for the other. In the mean time io package and core development have continued so this needs to be merged. I think any patches for tackling zip functions would need to be adapted to the merged functions.

Philip Nienhuis <philipnienhuis>
Group Member
Wed 05 Jul 2017 09:27:13 AM UTC, comment #57: 

I opened bug #51388 to track the issue with the "sluggishness" of "cd" on some systems.

@Philip: No hurry. I just wanted to make sure the other patch does not get lost as the scope of this bug report changed direction...
Since Octave packages get updates more regularly than core, are you still interested in working around that in io rather than waiting for an update of core Octave?

Markus Mützel <mmuetzel>
Group administrator
Wed 05 Jul 2017 06:24:10 AM UTC, comment #56: 

Yeah it would be a hard sell :-)

That leaves us with adapting unpack.m / unzip.m / gunzip.m.
Again I'd rather do this in a central place (i.e., core Octave) rather than for individual functions in an OF package.

BTW I mentioned before there is a similar bug report where this issue with zip was also mentioned and where (IIRC) Mike figured it would be a (fast SSD/HD) driver issue. But I failed to dig it up.
I'll cc him, hopefully he still remembers where it was.
(as it may help to tackle/close that bug report as well.)
...oh but he IS cc'd...

{slightly OT]
@Markus, now that I got some issues out of the way with mxe cross-builds I'll review and push your patch from comment #25, thanks for your patience :-)

Philip Nienhuis <philipnienhuis>
Group Member
Tue 04 Jul 2017 10:11:18 PM UTC, comment #55: 

InfoZIP is the default zip and unzip programs on all GNU systems, so these should be the exact same versions (3.0 and 6.0 respectively) that are on every GNU/Linux system out there, and are provided to mxe-octave via msys standard packages.

I think trying to get Octave core to use something other than the default zip and unzip programs to deal with zip format archives is kind of a hard sell.

The "while-strcmp-cd" loops in this patch look very suspect to me, there should be no situation where the pwd does not equal the directory that was just cd'd to. Are these loops actually necessary? Do we need to fix cd / chdir?

The rest of the patch I guess has to do with zipping and unzipping to a temporary file and then moving the result over the real destination file name.

Are there bug reports about why the pack and unpack functions are not good enough to do what is needed here?

Mike Miller <mtmiller>
Group Member
Tue 04 Jul 2017 06:25:52 PM UTC, comment #54: 

@Andrey and Markus:
Good detective work, thanks. It sounds very convincing to me - I've always wondered why it is on systems with a fast HD / SSD that we see this bug; your explanation fits this observation perfectly: such systems finished the actual write operations before infozip itself is ready.

I am not prepared (yet) to just apply your patch for the OCT interface:

  • All io pkg spreadsheet interfaces that rely on zip need a similar patch.
  • There's also UNO (AFAIK relying on gzip) that has similar issues.


As I have a hard time believing that issues with zip / infozip are limited to the io package I'd suggest to bring this (= replacing infozip and/or fixing unpack.m) up in the maintainers ML.

Philip Nienhuis <philipnienhuis>
Group Member
Tue 04 Jul 2017 08:48:04 AM UTC, comment #53: 

Thanks to a lot of trial and error with Andrey's help, we could pin the second issue down to some strange behavior of InfoZip:
InfoZip relies on the current working directory (cwd) to function properly. On some systems (like on Andrey's and others), it looks like it takes some time to update the cwd properly. That does not seem to be handled by Octave's "unpack" function.
When calling InfoZip directly and taking special care that the cwd has changed properly, the behavior that Andrey observed disappears on his system.
The attached patch includes these changes.

Btw, we quickly checked with using 7-Zip (because we both had it at hand). That program accepts absolute paths for source and destination (unlike InfoZip that relies on the cwd for relative paths in the archive). Using 7-Zip, that issue disappeared as well (without any additional hassle).
I am still not sure why on some systems the issue with the slow cwd change exists in the first place. But we might consider using a more modern program than InfoZip (which dates back to 2008 for the zip part and 2009 for the unzip part) that better handles those systems.

PS: The patch for the other issue (transcode to UTF-8 from any supported code page) is in comment #25 of this already quite long discussion.

(file #41114)

Markus Mützel <mmuetzel>
Group administrator
Wed 21 Jun 2017 02:53:30 PM UTC, comment #52: 

when I deleted ";" in your _OCT:

Checking requested interface(s):
OCT*; (* = default interface)
tmpdir = C:\Users\A2731~1.ALE\AppData\Local\Temp\oct-PJzzsG
ext = .xlsx
templ = C:\OCTAVE~1.1\share\octave\packages\io-2.4.7\templates\template.xlsx
opwd = C:\05octave
unp_tic = 1498056683948
file C:\05octave\o076xls_sm.xlsx couldn't be unpacked. Is it the proper file format?
xls = [](0x0)
warning: __OCT_spsh_open__: some elements in list of return values are undefined
warning: called from
    __OCT_spsh_open__ at line 71 column 7
    xlsopen at line 318 column 30
    xlswrite at line 202 column 9
    o076xls_sm at line 15 column 1
warning: xlsopen.m: file type not supported by oct
warning: called from
    xlsopen at line 338 column 7
    xlswrite at line 202 column 9
    o076xls_sm at line 15 column 1


Andrey Aleshin Igorevich <iamandreyaleshin>
Wed 21 Jun 2017 02:24:51 PM UTC, comment #51: 

Output file is absent always (xlsx)

Andrey Aleshin Igorevich <iamandreyaleshin>
Wed 21 Jun 2017 02:06:00 PM UTC, comment #50: 

your correction in c:\octave-4.2.1\share\octave\packages\io-2.4.7\private\__OCT_spsh_open__.m

unp_tic = tic;
while toc (unp_tic) < 2 && ...
  ! exist (fullfile (tmpdir, unp_files{end}), "file")
 pause (0.2);
endwhile


script:

clear; clc;
cell1={'One, two, three, 4'};
filename=[mfilename('fullpath'),'.xlsx'];
if exist(filename)==2
  delete(filename);
end
%pause(0.05);
xlswrite(filename,cell1,[],[],'oct');


Andrey Aleshin Igorevich <iamandreyaleshin>
Wed 21 Jun 2017 01:58:52 PM UTC, comment #49: 

Of course. It is need to reopen Octave to clean results, otherwise - it works from cache:

Checking requested interface(s):
OCT*; (* = default interface)
file C:\05octave\o076xls_sm.xlsx couldn't be unpacked. Is it the proper file format?
warning: __OCT_spsh_open__: some elements in list of return values are undefined
warning: called from
    __OCT_spsh_open__ at line 71 column 7
    xlsopen at line 318 column 30
    xlswrite at line 202 column 9
    o076xls_sm at line 15 column 1
warning: xlsopen.m: file type not supported by oct
warning: called from
    xlsopen at line 338 column 7
    xlswrite at line 202 column 9
    o076xls_sm at line 15 column 1


Andrey Aleshin Igorevich <iamandreyaleshin>
Wed 21 Jun 2017 01:24:58 PM UTC, comment #48: 

Andrey, could you please find the folder where "__OCT_spsh_open__.m" is and replace it with the one I attached?
This folder is for me: C:\Octave\Octave-4.2.1\share\octave\packages\io-2.4.7\private

Can you try whether you can still reproduce the issue and post any error messages or warnings you see at the command prompt?

(file #41012)

Markus Mützel <mmuetzel>
Group administrator
Wed 21 Jun 2017 05:44:39 AM UTC, comment #47: 

May be at the empty folder "delete" command not expend time to search a file, work quickly and release the hdd.

Otherwise at the full folder "delete" command expend time to searching file among other but time is continue and next command "xlswrite" attempt to work and collision is appear.

I know my thinks, words are not professionally but they are. (I have no more time to study in depth your subject now). I viewed you understood me. Thank you!
--
About second bug - xlsx. Thank you too for understanding and for a patch. I will try compiling development packages later.

I don't want to use your time in vain. Therefore I think it is the part of end of topic. If I recieve new results I will write here.

Have a good day. Thank you very much both (Philip Nienhuis and Markus Mützel). I discovered new thinks for me.
Best regards

Andrey Aleshin Igorevich <iamandreyaleshin>
Wed 21 Jun 2017 05:12:29 AM UTC, comment #46: 

You said in detail. Thank you

Andrey Aleshin Igorevich <iamandreyaleshin>
Tue 20 Jun 2017 06:36:04 PM UTC, comment #45: 

Andrey,

The bug you are referring to now (apparently unreliable writing to file) has been reported before (can't find it now, was about 3-4 months back IIRC). It has to do with very fast hard disks. I hit this already 3-4 years ago.

Mike Miller and John Eaton figured it would be a bug in hardware drivers, but now I think it is really a bug in zip (that zips the various XML files together, .xlsx is a mere zip archive):
 

  • zip hands over the actual writing of the zip file to disk to lower level routines and returns.
  • The lower level routines may take their time to do the actual writing.
  • Next however, as zip has returned, unzip is invoked and it tries to read the .xlsx file that hasn't been finished yet.


I think zip should return only after having verified that its output file has been finished completely.

In the io_xls_testcript.m and io_ods_testscript.m function files in the io package you can see that I've entered various pause commands to separate xlswrite and xlsread (odswrite and odsread) calls. Those delays are there for the very same reason you complain about.

Philip Nienhuis <philipnienhuis>
Group Member
Tue 20 Jun 2017 03:07:39 PM UTC, comment #44: 

Re: It would be easier to follow if you post the errors and warning directly as text.

Ok. I will write warnings and errors here. Thank you




As Philip said:
Repeating of xlswrite until result or this pause can do stable output by me.

Me very pleasantly your both attention to me. But I want say I try to recognize this bugs to help you both to write Octave's library and Octave 4.4.

It's not a problem that stopped my work. But I wanted to help you.

Philip said has no time to me today but will have later. And I didn't want to divert him today. I wrote to the future

I hope you receive my bug's results in later time. I don't think the reason in my computer and not need to correct whole Octave with pause only for me. Therefore I try to understand this bug many times

Thanks very much. I will absent until tomorrow

Andrey Aleshin Igorevich <iamandreyaleshin>
Tue 20 Jun 2017 02:51:30 PM UTC, comment #43: 

Those files aren't really strange. That is what the more recent MS Office files are: zip-files with a certain structure.

It would be easier to follow if you post the errors and warning directly as text.

@Philip: There indeed seem to be two instances of "return" in _OCT_spsh_open_ (and at least in _COM_spsh_open_, too) where the last output argument "lastintf" is not yet set. I will leave that error to you because I am not sure what that argument is used for and what should be returned in those cases.

There seems to already be a manual pause after unpack in line 63:

      unpack (templ, tmpdir, "unzip");
      ## Allow unpack dust to settle down (lazy write file I/O)
      pause (0.2);


It might be that this pause is too short on Andrey's system and the file I/O needs more time to flush. Maybe waiting some time whilst checking for the last file in the return value of unpack to be available might be necessary? Any better ideas?

Markus Mützel <mmuetzel>
Group administrator
Tue 20 Jun 2017 02:43:45 PM UTC, comment #42: 

It's an example. This bug appeared when I wrote a middle size script. I change my script, run it and view results in xlsx. But sometimes I changed my script, ran it, but result was absent

Andrey Aleshin Igorevich <iamandreyaleshin>
Tue 20 Jun 2017 02:38:19 PM UTC, comment #41: 

result is same on another computer. Stable non-assembled xlsx after  1-4 attempts to run with 1-9 folders and successfully in empty.

(file #40957)

Andrey Aleshin Igorevich <iamandreyaleshin>
Tue 20 Jun 2017 02:06:47 PM UTC, comment #40: 

You can take any xlsx file. Change it file type from .xlsx to .zip and unpack it to new directory. You will see same strange files as I sent you

Andrey Aleshin Igorevich <iamandreyaleshin>
Tue 20 Jun 2017 02:01:56 PM UTC, comment #39: 

I attached pictures.

It's work very quickly with 'oct'.

HDD is flash sata2. Very quick.

Strange situation is:
All is correct in empty work directory and xlsx dissamble by parts when work dir is full.

This strange files is a part of one xlsx file, but I don't can assemble it manual. It need to do by Octave automatically.

Files attached. Their contents same as in dissasembled xlsx but they are not a strong clone.

I'm sorry, I forgot to attach warning last time

file #40956)

Andrey Aleshin Igorevich <iamandreyaleshin>
Tue 20 Jun 2017 01:43:53 PM UTC, comment #38: 

Still cannot reproduce here even with bigger files.

Good to here that the short pause solves the issue for you.
If that is helping, the issue might be due to a busy or slow hard disk when the file is created.
What does system("set temp") return for you? Is that a local drive?

If you do not see any errors or warnings in the command window, there is probably little we can do to help.

Markus Mützel <mmuetzel>
Group administrator
Tue 20 Jun 2017 12:12:40 PM UTC, comment #37: 

Dear Markus Mützel,
Is I wrote that you request me?

It's not an utf bug, but this bug I described in first message

Andrey Aleshin Igorevich <iamandreyaleshin>
Tue 20 Jun 2017 12:05:52 PM UTC, comment #36: 


"pause(0.05);"


all correct.
I don't know how this delete's command affect to no stable output, but pause is an alternative soluition.

addiction:
bug is absent when use with 'com' interface .

output script is:

clear; clc;
cell1={'One, two, three, 4'};
filename=[mfilename('fullpath'),'.xlsx'];
if exist(filename)==2
  delete(filename);
end
pause(0.05);
xlswrite(filename,cell1,[],[],'oct');


Andrey Aleshin Igorevich <iamandreyaleshin>
Tue 20 Jun 2017 12:04:05 PM UTC, comment #35: 

I'm sorry for long time. I don't know a reason, but I found an alternative solution. Below about it.

Well.
Octave 4.2.1 from zip: https://ftp.gnu.org/gnu/octave/windows/octave-4.2.1-w64.zip without any patches (had no time to try their)

io *|   2.4.7
windows *|   1.2.4:

Initial conditions:
empty work folder (05octave to me)

Close and open your Octave.

We create a new script: o076xls.m
with this content:


clear; clc; close all; %help;
cell1={'One, two, three, 4'};
filename=[mfilename('fullpath'),'.xlsx'];
if exist(filename)==2
  delete(filename);
end
xlswrite(filename,cell1,[],[],'oct');


and run it by pressing F5.

Running it many times it do right output xlsx.

Then we execute manually this command:

system('mkdir 1 2 3 4 5 6 7 8 9');

to create 9 folders (this is an example a non-empty folder with main script).

Then run our script by pressing F5 many times. 3-4

And sometimes we have no an output file xlsx, but have some parts of it.

I found just now:
If we write a pause:

pause(0.05);
-verbatim

all correct.
I don't know how this delete's command affect to no stable output, but pause is an alternative soluition.

addiction:
bug is absent when use with 'com' interface .


(file #40948, file #40949, file #40950, file #40951)

Andrey Aleshin Igorevich <iamandreyaleshin>
Tue 20 Jun 2017 11:12:15 AM UTC, comment #34: 

Thanks. I doing it now. 10-15 minutes to write steps

Andrey Aleshin Igorevich <iamandreyaleshin>
Tue 20 Jun 2017 11:05:19 AM UTC, comment #33: 

I cannot reproduce this here. But I cannot read Russian, so I might have gotten some steps wrong. Can you please provide easy steps in English if possible with the code you are running and what warning/errors you see in a new bug report.
Please, keep this report dedicated to the Unicode issue with xlswrite.

Markus Mützel <mmuetzel>
Group administrator
Tue 20 Jun 2017 09:20:49 AM UTC, comment #32: 

I posted in pictures:
I don't know, how to assemble this strange files in one, but I understand they are parts of file xlsx that I waited.

Bug in this:
in empty workspace xlxs is created, in full directory - not always



Andrey Aleshin Igorevich <iamandreyaleshin>
Tue 20 Jun 2017 08:47:31 AM UTC, comment #31: 

Good day to all! It's a good day really.

I found conditions my main bug and this behavior's description. I am posting now (not with encoding - another bug - first bug of this topic)

Andrey Aleshin Igorevich <iamandreyaleshin>
Thu 15 Jun 2017 08:26:53 AM UTC, comment #30: 

Thank you both

Andrey Aleshin Igorevich <iamandreyaleshin>
Wed 14 Jun 2017 07:00:48 PM UTC, comment #29: 

Markus, thanks for the patch.
I'll review and probably push but I'm busy with other things now.

Philip Nienhuis <philipnienhuis>
Group Member
Wed 14 Jun 2017 10:11:04 AM UTC, comment #28: 

Thank you. I will try it if I can. I will read their

Andrey Aleshin Igorevich <iamandreyaleshin>
Wed 14 Jun 2017 09:05:42 AM UTC, comment #27: 

Andrey, that patch applies to the Mercurial repository of the io package. Philip who is its maintainer might review and push it there.
Anyway - like I already wrote - the patch would help only with the current development version of Octave which will eventually become version 4.4. You could test that by cross compiling Octave for Windows.
If you are interested in helping to test the next version of Octave, you could start by reading the Wiki pages:
http://wiki.octave.org/Building
http://wiki.octave.org/Windows_Installer

Markus Mützel <mmuetzel>
Group administrator
Wed 14 Jun 2017 05:54:06 AM UTC, comment #26: 

Thank you. Is I correct try to use a patch? Please, see a attached screen.

I found in Internet next instructions:
#! /bin/sh
#
# To apply this patch, cd to the top level Octave source directory and
# run this file through /bin/sh.  It will first remove any files that
# have been deleted from the source distribution since the last
# release and then update the sources with patch(1)...

Is same actions need in Windows?
I use: bin\sh.exe bug51203_xls_unicode.patch
I added "#" in line 7.

Is need to run via Windows Octave-editor? I'm sorry for this simple question.


Andrey Aleshin Igorevich <iamandreyaleshin>
Tue 13 Jun 2017 08:12:44 PM UTC, comment #25: 

Attached is a patch for Octave IO that should fix the issue on current development builds. I only tested with system code page CP1250 but it should work with other system code pages, too.

(file #40910)

Markus Mützel <mmuetzel>
Group administrator
Tue 13 Jun 2017 06:27:11 PM UTC, comment #24: 

Very thanks, Markus Mützel. Me pleasantly you are thinking about it.
It is not a problem by me today. I write English only ascii 32-127 scripts today.
I have a some perfectionist's behavior and I like to do a work perfect. Therefore I interest it. Because native characters are very friendly to default user.

I will wait Octave 4.4 patiently. I think I will understand some important features during and until an it's creation. May be it will be useful to me or other.

I view this forum and will view. I understand you. Thank you!

Andrey Aleshin Igorevich <iamandreyaleshin>
Tue 13 Jun 2017 05:04:22 PM UTC, comment #23: 

Andrey, the behavior that you are describing is due to the fact that Excel needs the text encoded in UTF-8. If you save your file in UTF-8 and do not mangle with the strings, Excel is happy.
However as you wrote, that does not match Octave's nor Matlab's default encoding (and you might hit other bugs like bug #51210).
As of Octave version 4.2, there is no Octave function to manually convert between arbitrary encodings. Octave 4.4 will contain "native2unicode" and "unicode2native" which could be used for this. Unfortunately, there is no way I can think of to solve your issues at the moment (other than manually writing a function that converts strings between your local encoding "Windows-1251" and UTF-8).

Markus Mützel <mmuetzel>
Group administrator
Tue 13 Jun 2017 03:15:54 PM UTC, comment #22: 

Good day.

I found next:

Linux & Oct work ideal - plots, xls-es.

If in Windows in Octave to set editor encoding UTF-8 and to write script, It will be in UTF-8. And it do output in xlsx (...'oct') fine as said Markus Mützel. Thanks. But this script incompatible with default users.



But. If to write script in UTF-8, for example in Notepad++, to convert, to save in UTF-8 and to open in Windows Octave, then file will be not readable by me, but it correctly do output in Excel w/o flag "convert_utf". Text is readable.

But if you to run UTF-8-script in Windows-Matlab, then you can see in Excel same characters as in Matlab/Octave editor.

I don't can to understand it now. I see, that Matlab to do output this character, that view.

But Octave to do output character that it load. But it don't display it. That is: There are a script in UTF-8. Octave with 1251's editor don't may to display it correctly. But It do output exactly as scriptt saved.
Although, it must to do output that it display. In my opinion.

I'm sorry for my long comments and my write style. I will think further.


Andrey Aleshin Igorevich <iamandreyaleshin>
Sun 11 Jun 2017 02:06:11 PM UTC, comment #21: 

Title adapted

Philip Nienhuis <philipnienhuis>
Group Member
Fri 09 Jun 2017 11:59:30 AM UTC, comment #20: 

I can confirm Andrey's observation regarding text in graphics. I opened a separate report for this issue in bug #51210.

Markus Mützel <mmuetzel>
Group administrator
Fri 09 Jun 2017 09:33:46 AM UTC, comment #19: 

All right. I am a new user here. I'm studying behavior on this site.

Yes, I use alternatively way to output by repeating by cycle today.

Your ideas and reflections are useful by me.

I found, that UTF8 encoding by Octave editor allow to do output via 'oct' wide-ASCII characters correct. utf8.m files are opened on Linux without troubles on Linux in any math soft (because Linux has utf8, I think). In Windows Matlab utf8.m file don't readable.

I use Octave to write .m files, but default users may use both soft in both or in three Operating Systems. Therefore this solution may be using by me today, but this is not universal.
And, when I changed Octave-Editor locale to utf-8 by default - Plotting fonts don't display normally.
I will to study encoding problem now, but there is available to write English scripts only by me to current tasks today.
--
Yea. I accept your action - to set status "postponed".

I'm sorry that my answers are too long.

Thank you all very much. Me pleasantly for your time and thoughts. You can write me later in any time. I will answer and I will to try any your cases and examples.

With best wishes


Andrey Aleshin Igorevich <iamandreyaleshin>
Fri 09 Jun 2017 08:55:58 AM UTC, comment #18: 

Dear Andrey,

Currently I lack time to dive into your issues more thoroughly, but I will do that later.
But I understand you have a workaround so for the time being I'm lowering priority of this bug report and set status to "postponed" as a real fix (unicode processing) will be implemented later on.

Philip Nienhuis <philipnienhuis>
Group Member
Fri 09 Jun 2017 08:08:00 AM UTC, comment #17: 

But, Thank you very much, that you solve my problem with encoding.
I will to study it soon today or in the coming days too.

Best regards.

Andrey Aleshin Igorevich <iamandreyaleshin>
Fri 09 Jun 2017 06:29:03 AM UTC, comment #16: 

Good morning.

Very thanks, that you are discussing about them. I don't carefully understand all information, that you wrote to each other with 49222, I've read it, but I understand a common idea and I will understand it.

Do I understand correctly, that future (and being developed now) Octave 4.4 will contain native2unicode and unicode2native functions?

In Matlab they work non ideal with all characters but work.

If Octave will contain their, I think it will be good. Although how binary data in unicode write in cell when transfer to xls/xlsx? But it will be good in any case.

I found 'UTF-8' encoding in Octave. It's one of solutions. Although this code is not readable by Matlab users. Well.

Thanks very much you both.
It about encoding.



How about this no stable output with 'oct', when other files or/and folders are exist? It's interested by me first. And I think, that this is not small inaccuracy, but this is a more important bug. Both: xlswrite and xlsopen-oct2xls-xlsclose don't do a guaranteed output, when another folders/files are.





(file #40890)

Andrey Aleshin Igorevich <iamandreyaleshin>
Thu 08 Jun 2017 05:26:33 PM UTC, comment #15: 

@Markus:
yes I'd appreciate cooperation.
I knew the limitations of what is now implemented in the io package.
Note that only some of the available spreadsheet I/O interfaces need conversion, the others do it OOTB.

I've seen native2unicode and its sibling but they consistently give errors. As soon as that is solved we can proceed.

Philip Nienhuis <philipnienhuis>
Group Member
Thu 08 Jun 2017 05:05:15 PM UTC, comment #14: 

Andrey,

Working with encodings is a mess. Especially on Windows.
To make your live easier, I would always use UTF-8 with Octave. To change your default encoding in the editor, please open your preferences (menu "Edit" -> "Preferences"). On the very bottom on the "Editor" tab, select "UTF-8" from the drop-down list.

Now close all open tabs in the editor. When you open your m-files again, you might notice that your Cyrillic characters look messed up. You can either convert your m-files to UTF-8 (e.g. with Notepad++) or correct the messed up characters manually.

With these settings, the following example works for me:

cell1={'Анализ : ddmmyyyy_hhmmss'};
cell1(end+1,1)={1};
cell1(end,2)={2};
cell1(end,3)={3};

cell1(end+1,1)={'name'};
cell1(end,2)={'min'};
cell1(end,3)={'max'};

xls = xlsopen ("o076xls_utf8.xlsx", 1, "oct");
xls = oct2xls (cell1, xls, 1, "");
xls = xlsclose (xls);

xlswrite ("o0776xlswrite.xlsx", cell1);


Do not use the "convert_utf" setting in this case. Your input is already UTF-8 encoded.



Philip,

The dirty hack that we used to solve bug #49222 only works for encodings which match Unicode in the 128-255 area. That means it only works for "Latin-1" (or to some extend "ANSI") encoding. I did not realize this at the time (sorry).
The current dev builds include "native2unicode" and "unicode2native". If you like, we could work on integrating these functions to make the io package work with all supported encodings when Octave 4.4 comes out.

Markus Mützel <mmuetzel>
Group administrator
Thu 08 Jun 2017 02:37:42 PM UTC, comment #13: 

I'm sorry, I read ASCII 128-255 localized by each language as I understand

Andrey Aleshin Igorevich <iamandreyaleshin>
Thu 08 Jun 2017 02:10:22 PM UTC, comment #12: 
Andrey Aleshin Igorevich <iamandreyaleshin>
Thu 08 Jun 2017 01:44:51 PM UTC, comment #11: 

Russian letters are char codes: 168 and 192-255, that wider than [32:127] and I think, their need to convert as says in https://octave.sourceforge.io/io/function/oct2xls.html but today converting is not right. It need to think about it at second time

Andrey Aleshin Igorevich <iamandreyaleshin>
Thu 08 Jun 2017 01:25:13 PM UTC, comment #10: 

I don't think, that my solution is correct. And I thought, that I must to notify a world as says in https://octave.sourceforge.io/bugs.php

Thanks.

Andrey Aleshin Igorevich <iamandreyaleshin>
Thu 08 Jun 2017 12:59:21 PM UTC, comment #9: 

Yea, Andrey

Andrey Aleshin Igorevich <iamandreyaleshin>
Thu 08 Jun 2017 12:56:55 PM UTC, comment #8: 

Is my English so bad? I'm sorry. I tell only in forums and shops. I try to speak better. I'm very sorry again.
--
I understood about Linux and utf8. Thanks. I will remember it feature. I will try to solve an encoding problem later.
('com'-interface in windows correct do output in xls file, but don't do in xlsx. In Linux com interface don't work as I think, and oct don't output in xls.)
--
I don't touch this xlsx file during running script. I understand it.

I solve this task by using 'oct'-interface and repeating output again and again while file is not exist or counter t<=3.
--
I try to use xlsopen->oct2xls->xlsclose, but result is same as xlswrite:

When work folder with script(which contain oct2xls or xlswrite) is empty. The script work ideal.
When I create 9 folders (for example:)

system('mkdir 1 2 3 4 5 6 7 8 9')

a result is not guaranteed. After 1 times, after 2-5 times, my script don't output xlsx-file, displays warnings or errors and create strange xml-folders and files.

If I continue and I press F5 to run script. It will be work and do output to xlsx.
--
Thank you very much for answer. You can say me any words.

Andrey Aleshin Igorevich <iamandreyaleshin>
Thu 08 Jun 2017 12:29:20 PM UTC, comment #7: 

Sorry Aleshin (or is it Andrey?) I may not understand you very well; I'm sorry I don't know better Russian :-)
Is your problem solved now?

The warning:

:
warning: xls2open: file C:\05octave\o076.xlsx couldn't be unzipped
:


is telling and may be caused by having o076.xlsx opened in another program so that it is locked and cannot be overwritten.
Did you try with another filename?

It does not surprise me that you do not need the "convert-utf" option on Linux. The Octave terminal on Linux is a better one than the Octave terminal that is used on Windows; therefore on Linux double-byte chars are probably treated much better by Octave so that they do not need treatment with the "convert-urf" option.

Philip Nienhuis <philipnienhuis>
Group Member
Thu 08 Jun 2017 12:22:58 PM UTC, comment #6: 

while (exist(filename)~=2)&&t<=3

Andrey Aleshin Igorevich <iamandreyaleshin>
Thu 08 Jun 2017 11:58:36 AM UTC, comment #5: 

xlswrite(filename,cell1,'sh1','A1');

t=1;
while (exist(filename)~=2)&&t<=1
  xlswrite(filename,cell1,[],[],'oct');
  t=t+1;
end

if (exist(filename)~=2)
  fprintf('Please repeat script\n');
end

------
This is not liked by me, but today it is works only that. Me smile.

Andrey Aleshin Igorevich <iamandreyaleshin>
Thu 08 Jun 2017 11:57:16 AM UTC, comment #4: 


xlswrite(filename,cell1,'sh1','A1');

t=1;
while (exist(filename)~=2)&&t<=1
  xlswrite(filename,cell1,[],[],'oct');
  t=t+1;
end

if (exist(filename)~=2)
  fprintf('Please repeat script\n');
end
-verbatim
This is not liked by me, but today it is works only that. Me smile.

Andrey Aleshin Igorevich <iamandreyaleshin>
Thu 08 Jun 2017 11:55:54 AM UTC, comment #3: 


xlswrite(filename,cell1,'sh1','A1');

t=1; %'oct' не всегда выводится с 1 раза
while (exist(filename)~=2)&&t<=1 %пока не сущ и к-во поп<=3
  xlswrite(filename,cell1,[],[],'oct');
  t=t+1;
end

if (exist(filename)~=2)
  fprintf('Please repeat script\n');
end
-verbatim
This is not liked by me, but today it is works only that. Me smile.

Andrey Aleshin Igorevich <iamandreyaleshin>
Thu 08 Jun 2017 11:54:54 AM UTC, comment #2: 

Me pleasantly, that you answered me. Thank you very much. Thank you very much. I studied, understood and tried new variant a long time, therefore I write only now.

I replace xlswrite by oct2xls as you recommended.

Russian symbols with 'oct' output correct only in Linux without structure "convert_utf". Me smile. I attached screenshots and xlsx-es.

But this is not priority task. Today, it is may to read and write in English only.
-------------
Main task - is to do sure output in xlsx - was not obtain by me with oct2xls. In Linux - no problems. Sure and guaranteed. In Windows, I have same result. Therefore, problem may be is not be related with xlswrite and oct2xls. I don't know, what.

In two words: when script to be with files and folders - often there are xml-files instead file.xlsx.

Yea, io-2.4.7:

pkg list
Package Name         | Version | Installation directory
---------------------+---------+-----------------------
                 io *|   2.4.7 | C:\OCTAVE~1.1\share\octave\packages\io-2.4.7


oct2xls:

filename = C:\05octave\o076.xlsx
warning: xls2open: file C:\05octave\o076.xlsx couldn't be unzipped
warning: __OCT_spsh_open__: some elements in list of return values are undefined
warning: called from
    __OCT_spsh_open__ at line 85 column 7
    xlsopen at line 318 column 30
    o076xls at line 25 column 5
warning: xlsopen.m: file type not supported by oct
warning: called from
    xlsopen at line 338 column 7
    o076xls at line 25 column 5
error: oct2xls: invalid xls file pointer struct


xlswrite:

filename = C:\05octave\o076.xlsx
warning: xls2open: file C:\05octave\o076.xlsx couldn't be unzipped
warning: __OCT_spsh_open__: some elements in list of return values are undefined
warning: called from
    __OCT_spsh_open__ at line 85 column 7
    xlsopen at line 318 column 30
    xlswrite at line 202 column 9
    o076xls at line 36 column 3
warning: xlsopen.m: file type not supported by oct
warning: called from
    xlsopen at line 338 column 7
    xlswrite at line 202 column 9
    o076xls at line 36 column 3


Today I use a not right solution - to repeat output:

xlswrite(filename,cell1,'sh1','A1');

t=1; %'oct' не всегда выводится с 1 раза
while (exist(filename)~=2)&&t<=1 %пока не сущ и к-во поп<=3
  xlswrite(filename,cell1,[],[],'oct');
  t=t+1;
end

if (exist(filename)~=2)
  fprintf('Please repeat script\n');
end
-verbatim
This is not liked by me, but today it is works only that. Me smile.

(file #40886, file #40887, file #40888, file #40889)

Andrey Aleshin Igorevich <iamandreyaleshin>
Thu 08 Jun 2017 09:24:53 AM UTC, comment #1: 

The issue is that you're being hit by double-byte character issues; I'm sorry for that.  Octave isn't very good at handling double-byte characters esp. on Windows.
Luckily there's a way out for you.

First, be sure to upgrade the io package to 2.4.7 (latest version).
Then in your script change the line:

xlswrite(filename,cell1,[],[],'oct');

into:

xls = xlsopen ("o076xls.xlsx", 1, "oct");
xls = oct2xls (cell1, xls, 1, "", struct ("convert_utf", 1));
xls = xlsclose (xls);

and try again.

If I do that I get the attached file that is read perfectly fine by Excel and LibreOffice.

FYI: the option "convert_utf" is what you need. Try:
help oct2xls
or
help oct2ods
and
help xls2oct
or
help ods2oct
for more info.

Please report back!

Release => other (as it is an OF package issue and not core Octave)


(file #40885)

Philip Nienhuis <philipnienhuis>
Group Member
Thu 08 Jun 2017 07:12:03 AM UTC, original submission:  

Welcome.
function xlswrite with interface 'com' output in both formats: xls and xlsx. But an Excel don't open xlsx, send error "file corrupted".

with interface 'oct' in Windows: If variable contain Russian symbols, file is outputed, but Excel don't may to read this. It may attempt to open, but it view only some digits.

with interface 'oct' in Linux(Ubuntu) and LibreOffice: errors are absent. All right and correct.

But in Windows xlswrite sometimes not output data in file. Instead Octave send warnings listed below, and there are appear a strange folders and files:

_rels\
docProps\
xl\
[Content_Types].xml


When Octave's work directory is empty - all right. But, if no, for example, I create 9 folders. You can open a file with xlswrite(...'oct') and don't get output.
To view this bug, you can create a some amount folders, open m-file and run it. From 1 to 10 attempts - bug will appear guaranted



Checking requested interface(s):
OCT*; (* = default interface)
warning: xls2open: file C:\05octave\o076xls.xlsx couldn't be unzipped
warning: __OCT_spsh_open__: some elements in list of return values are undefined
warning: called from
    __OCT_spsh_open__ at line 85 column 7
    xlsopen at line 318 column 30
    xlswrite at line 202 column 9
    o076xls at line 23 column 1
warning: xlsopen.m: file type not supported by oct
warning: called from
    xlsopen at line 338 column 7
    xlswrite at line 202 column 9
    o076xls at line 23 column 1


Andrey Aleshin Igorevich <iamandreyaleshin>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #41820:  bug51203_xlsx_InfoZip_v4.patch added by mmuetzel (5KiB - application/octet-stream)
file #41242:  bug51203_xlsx_InfoZip_v3.patch added by mmuetzel (4KiB - application/octet-stream)
file #41219:  o076xls_with_check4.log added by iamandreyaleshin (188KiB - application/octet-stream)
file #41213:  bug51203_minor_fixes.patch added by mmuetzel (5KiB - application/octet-stream)
file #41206:  bug51203_xlsx_InfoZip_v2.patch added by mmuetzel (7KiB - application/octet-stream)
file #41114:  bug51203_xlsx_InfoZip.patch added by mmuetzel (7KiB - application/octet-stream)
file #41012:  __OCT_spsh_open__.m added by mmuetzel (7KiB - application/octet-stream)
file #40957:  o076xls.m added by iamandreyaleshin (257B - application/octet-stream)
file #40953:  001.png added by iamandreyaleshin (75KiB - image/png)
file #40954:  002.png added by iamandreyaleshin (15KiB - image/png)
file #40955:  003.png added by iamandreyaleshin (55KiB - image/png)
file #40956:  xlsx parts.7z added by iamandreyaleshin (3KiB - application/octet-stream)
file #40948:  01.png added by iamandreyaleshin (53KiB - image/png)
file #40949:  02.png added by iamandreyaleshin (37KiB - image/png)
file #40950:  03.png added by iamandreyaleshin (62KiB - image/png)
file #40951:  04.png added by iamandreyaleshin (59KiB - image/png)
file #40944:  scr20170620.jpg added by iamandreyaleshin (313KiB - image/jpeg)
file #40910:  bug51203_xls_unicode.patch added by mmuetzel (4KiB - application/octet-stream)
file #40909:  scr20170613.jpg added by iamandreyaleshin (148KiB - image/jpeg)
file #40892:  plot enc.jpg added by iamandreyaleshin (115KiB - image/jpeg)
file #40890:  screenshots OctEditorUTF8.7z added by iamandreyaleshin (115KiB - application/octet-stream)
file #40886:  m oct2xls.7z added by iamandreyaleshin (714B - application/octet-stream - use xlsopen->oct2xls-xlsclose instead xlswrite)
file #40887:  screenshots oct2xls.7z added by iamandreyaleshin (170KiB - application/octet-stream - use xlsopen->oct2xls-xlsclose instead xlswrite)
file #40888:  xlsx-es oct2xls.7z added by iamandreyaleshin (17KiB - application/octet-stream - use xlsopen->oct2xls-xlsclose instead xlswrite)
file #40889:  again same strange files oct2xls.7z added by iamandreyaleshin (3KiB - application/octet-stream - use xlsopen->oct2xls-xlsclose instead xlswrite)
file #40885:  o076xls-2.7z added by philipnienhuis (5KiB - application/octet-stream)
file #40880:  m.7z added by iamandreyaleshin (440B - application/octet-stream - not virus, not spam. Only full debug information)
file #40881:  xlsx-es.7z added by iamandreyaleshin (8KiB - application/octet-stream - not virus, not spam. Only full debug information)
file #40882:  screenshots.7z added by iamandreyaleshin (299KiB - application/octet-stream - not virus, not spam. Only full debug information)
file #40883:  strange files.7z added by iamandreyaleshin (3KiB - application/octet-stream - not virus, not spam. Only full debug information)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by jwe (Posted a comment)
  • -email is unavailable- added by mtmiller (Posted a comment)
  • -email is unavailable- added by mmuetzel (Posted a comment)
  • -email is unavailable- added by iamandreyaleshin (Submitted the item)
  • -email is unavailable- added by iamandreyaleshin (xlswrite. 'com' xlsx, 'oct' Russian cyr, 'oct' many attempts)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only group members can vote.

     

    Follow 25 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2018-07-02 jwe Summary[octave-forge] (io) xlsopen/xlsclose may create spurious files/folders instead of .xlsx file with &quot;OCT&quot; interface [octave forge] (io) xlsopen/xlsclose may create spurious files/folders instead of .xlsx file with "OCT" interface
    2017-09-24 philipnienhuis StatusNeed Info Fixed
        Open/ClosedOpen Closed
    2017-09-17 mmuetzel Attached File- Added bug51203_xlsx_InfoZip_v4.patch, #41820
    2017-09-03 philipnienhuis StatusPatch Submitted Need Info
    2017-08-11 jwe Summary[octave-forge] io xlsopen/xlsclose sometimes creates spurious files &amp;folders instead of .xlsx file with &quot;OCT&quot; interface [octave-forge] (io) xlsopen/xlsclose may create spurious files/folders instead of .xlsx file with "OCT" interface
    2017-08-11 jwe Summaryof-io: xlsopen/xlsclose sometimes results in spurious files and folders instead of .xlsx file with &quot;OCT&quot; interface [octave-forge] io xlsopen/xlsclose sometimes creates spurious files &folders instead of .xlsx file with "OCT" interface
    2017-07-19 mmuetzel Attached File- Added bug51203_xlsx_InfoZip_v3.patch, #41242
        Summaryxlswrite(...'com') output results in a corrupted .xlsx / unicode issues of-io: xlsopen/xlsclose sometimes results in spurious files and folders instead of .xlsx file with "OCT" interface
    2017-07-17 iamandreyaleshin Attached File- Added o076xls_with_check4.log, #41219
    2017-07-16 mmuetzel Attached File- Added bug51203_minor_fixes.patch, #41213
    2017-07-14 mmuetzel Attached File- Added bug51203_xlsx_InfoZip_v2.patch, #41206
    2017-07-04 mmuetzel Attached File- Added bug51203_xlsx_InfoZip.patch, #41114
    2017-06-21 mmuetzel Attached File- Added _OCT_spsh_open_.m, #41012
    2017-06-20 iamandreyaleshin Attached File- Added o076xls.m, #40957
    2017-06-20 iamandreyaleshin Attached File- Added 001.png, #40953
        Attached File- Added 002.png, #40954
        Attached File- Added 003.png, #40955
        Attached File- Added xlsx parts.7z, #40956
    2017-06-20 iamandreyaleshin Attached File- Added 01.png, #40948
        Attached File- Added 02.png, #40949
        Attached File- Added 03.png, #40950
        Attached File- Added 04.png, #40951
    2017-06-20 iamandreyaleshin Attached File- Added scr20170620.jpg, #40944
        Attached File- Added scr20170620_04.jpg, #40945

    Back to the top

    Powered by Savane 3.13-758e.
    Corresponding source code