bugGNU Octave - Bugs: bug #66288, [octave forge](io) Fails to import...

 
 

bug #66288: [octave forge](io) Fails to import from xlsx named ranges ending in numbers

Submitter:  None
Submitted:  Thu 03 Oct 2024 09:41:21 PM UTC
   
 
Category:  Octave Package Severity:  3 - Normal
Priority:  5 - Normal Item Group:  Incorrect Result
Status:  In Progress Assigned to:  philipnienhuis
Originator Name:  Ethan Zimmer Originator Email:  -email is unavailable-
Open/Closed:  * Open Release:  * other
Operating System:  * Any Fixed Release:  None
Planned Release:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Tue 15 Oct 2024 07:49:47 PM UTC, comment #7: 

OK thanks for answering to comment #4.

In the mean time I already have a fix along the lines of "best fix" in comment #4 but until November I lack time for rigorous testing with all interfaces.

As to Excel docs and names, the reality is somewhat more relaxed than you described. The largest column "number" in Excel is "XFD" (corresponding to real column number 16384) and it turns out a Named range "XFD1"  indeed isn't accepted but "XFE1" is - at least with my Excel2016.
I have no idea (yet) how Apache POI, LibreOffice and related SW handle this.

About the list of named ranges in the file pointer - yes as soon  as you enter a Named range range, xls2oct.m or oct2xls.m will get the list of Named ranges and add it to the xls pointer. So my reasoning when I first analyzed this bug report was "why not do that right away when opening the file", to save execution when invoking Named ranges. As it stands the execution flow in those parts of the code could be more efficient :-) if not more robust as well.

I'm unsure what you mean to say in your last paragraph in comment #4; but as regards scope, AFAICS in the expanded files and in the ECMA docs, each Named range can explicitly refer to one worksheet only.

I noted that Excel seems to maintain stale Named ranges, apparently pointing to deleted worksheets or so, I haven't investigated further. You'll recognize them by the '!REF#' entries in the range columns of the list of Named ranges in the xls pointer. Weeding out those could also make the whole show more robust, that is, at the Octave side - I have no intentions to "fix" such an .xlsx workbook.

Status => "In progress"

Philip Nienhuis <philipnienhuis>
Group Member
Tue 15 Oct 2024 03:48:17 PM UTC, comment #6: 

Thanks for this.  It is not urgent from my end. 

I've got no experience on .ods files, so I can't help with that.

Excel documentation says that cell references are disallowed as named ranges, so theoretically there should be no conflict.  I haven't tested that prohibition, and the maximum cell depends on version and maybe platform.  On my setup  it is XFD1048576 for xlsx files and IV65536 for .xls files.

I've noticed that a list of named ranges shows up occasionally in the xls pointer, but not when the file is opened.

One thing that might become important is the scope of the named range.  It is possible for a named range to exist for one worksheet in a file but not on any others. I haven't tested how Octave treats this, but the documentation seemed to ignore the issue.

Anonymous
Tue 15 Oct 2024 12:28:47 PM UTC, comment #5: 

Progress report:

The best fix would probably be to explore Named Ranges for all interfaces and file types (.xlsx, .ods, etc.) right away while opening a spreadsheet file and store them in the .xls pointer. That'll save a bit of overhead for each ensuing read action on the pertinent file.
But that needs quite a bit of work, esp. rigorous checking if all interfaces still work then.
Alternatively I can "just patch" the faulty file (private/chknmrange.m), but I'm afraid that also needs some thinking over to find the best thing to do.
I'd have time for either option only from November on.

If you urgently need a temporary fix I could upload a provisionally patched temporary (private) function file here that you can swap with the one in your Octave installation; but it would contain just a provisional kludge, without any guarantees that it won't introduce bugs elsewhere.

Philip Nienhuis <philipnienhuis>
Group Member
Sun 06 Oct 2024 07:43:53 PM UTC, comment #4: 

Hmm I thought I posted another comment yesterday (Saturday Oct 5).

Again:
Never mind comment #3, I can confirm the bug and it is present for all spreadsheet I/O interfaces.
The bug is that any range name resembling a one-cell spreadsheet range (like 'A3, 'BF24'; IOW alphanumeric characters followed by an integer number) will be interpreted as that one-cell range. Ridiculous as it may seem, 'popData2021' fits the bill :-)

Hmmmm this bug must have been there from the very first implementation of named ranges in the spreadsheet I/O in 2015. Intriguing that it has been hit only now.

I think I know how to fix this but only after next weekend I'll have time for it.

Philip Nienhuis <philipnienhuis>
Group Member
Sat 05 Oct 2024 10:08:03 PM UTC, comment #3: 

Thx, but again:

1. What spreadsheet I/O interface do you use?

Philip Nienhuis <philipnienhuis>
Group Member
Fri 04 Oct 2024 09:27:03 PM UTC, comment #2: 

chk_spreadsheet_support returns 0



(file #56474, file #56475)

Anonymous
Fri 04 Oct 2024 10:52:43 AM UTC, comment #1: 

1. What spreadsheet I/O interface do you use?
To see that, just run the command:
chk_spreadsheet_support
(w/o semicolon after the command)
in Octave. It'll return a number, that's enough for me.

2. Could you please upload a testfile displaying the issue?

Thanks.

Philip Nienhuis <philipnienhuis>
Group Member
Thu 03 Oct 2024 09:41:21 PM UTC, original submission:  

I'm using the io package to import data from Excel *.xlsx files.
Windows 10, 22H2, 64 bit.  Excel 2007 installed.

It works as expected when the named range ends in a character. For Example,
[TestData test flag] = xls2oct(test,'DOR Format','Names')

and

[TestData test flag] = xls2oct(test,'DOR Format','DOR2Census')

both work as expected.

But

[TestData test flag] = xls2oct(test,'DOR Format','popData2018')

does not work.  It returns an empty 0x0 cell structure, but the flag = 1, as if it worked.

[TestData test flag] = xls2oct(test,'DOR Format','popData2022')

also returns an empty cell structure, but it should return an error message since that named range doesn't exist in the spreadsheet.

[TestData test flag] = xls2oct(test,'DOR Format','popData')

returns data, but it shouldn't since that named range doesn't exist.  The data it returns is from 'popData2015', which is the lowest numbered popData range on the spreadsheet.

If I rename the range in Excel to end in a character, such as 'popData2021x', then it seems to work correctly.  To get this to work I need to save the xlsx file, then xlsclose the file in Octave, then re-open it with xlsopen.

[TestData test flag] = xls2oct(test,'DOR Format','popData2021x')

works as expected, returning correct data, and
[TestData test flag] = xls2oct(test,'DOR Format','popData2022x')

correctly gives an error message that the named range doesn't exist.

This is the work around that I am implementing.  I thought that there should be some description of this issue so the next person  doesn't have to spend as much time as I did figuring out how to make this work.




Anonymous

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #56474:  Test66288.m added by None (1KiB - application/octet-stream - chk_spreadsheet_support returns 0)
file #56475:  TestFile.xlsx added by None (12KiB - application/vnd.openxmlformats-officedocument.spreadsheetml.sheet - chk_spreadsheet_support returns 0)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by None (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only group members can vote.

     

    Follow 9 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2024-10-15 philipnienhuis StatusConfirmed In Progress
    2024-10-07 philipnienhuis SummaryFails to import from xlsx named ranges ending in numbers [octave forge](io) Fails to import from xlsx named ranges ending in numbers
    2024-10-06 philipnienhuis StatusNeed Info Confirmed
        Assigned toNone philipnienhuis
        Operating SystemMicrosoft Windows Any
    2024-10-04 None Attached File- Added Test66288.m, #56474
        Attached File- Added TestFile.xlsx, #56475
    2024-10-04 philipnienhuis Release9.2.0 other
    2024-10-04 philipnienhuis StatusNone Need Info

    Back to the top

    Powered by Savane 3.14-3b9d.
    Corresponding source code