bugGNU Octave - Bugs: bug #58080, [octave forge] (io) dbfwrite /...


bug #58080: [octave forge] (io) dbfwrite / shapewrite error

Submitted by:  Jaime Almonacid-Caballer <jaimepi>
Submitted on:  Mon 30 Mar 2020 07:50:26 AM UTC  
Category:  Octave Forge Package Severity:  3 - Normal
Priority:  5 - Normal Item Group:  Incorrect Result
Status:  Invalid Assigned to:  None
Originator Name:  JaimePi Open/Closed:  Closed
Release:  other Operating System:  GNU/Linux

Add a New Comment (Rich Markup)

You are not logged in

Please log in, so followups can be emailed to you.


( Jump to the original submission)

Thu 02 Apr 2020 07:54:21 AM UTC, comment #8: 

OK, thanks for reporting.

It wasn't all in vain as I hit & could improve a few things in the mapping package (e.g., replacing calls to deprecated functions in dev Octave, style, etc.)

As to your situation, yeah, that matches mine quite well :-) I'm about the only one in my company using Octave. It's always been a hard sell esp. considering graphics performance and lack of classdef I/O. We're moving away from Matlab/Octave anyway to Python and some R. QGIS is being used more and more in Azure although our IT dept. declared it more or less illegal ("incompatible" - hear hear, project files from every ArgGIS minor release upgrade are usually incompatible with its predecessors)
Good luck!

Closing report.

Philip Nienhuis <philipnienhuis>
Project Member
Thu 02 Apr 2020 07:10:06 AM UTC, comment #7: 

Dear Philip,
Thanks for your implication. Yes, it is a bit desperate situation. I am the first in my family/friends/OFFICE trying to switch from windows to linux, from matlab to octave, from ArcMap to QGIS and it is not being easy. I am forced to demonstrate that, with no big effort, everything can be done in this "new" collaborative environment.

That is the reason I use Octave 4.2.2 because it is that given directly without flatpak.

I have run the test with the Octave available via flatpak (Octave 5.2) and I have had no problem so I think there is no problem anymore.

Next step, make Octave/QGIS/GDAL to read multiband jp2 images properly.

Really, thanks for your patience. :)

Jaime Almonacid-Caballer <jaimepi>
Wed 01 Apr 2020 09:29:16 PM UTC, comment #6: 

In Mageia-7 Linux (x86_64) I still get the same results.

The only difference between Testinput.dbf and the output .dbf is in the column header meta data, only LibreOffice indicates that; QGIS still reads both files fine.

In your test.m script you compare the .dbf files written by both shapewrite and dbfwrite. They should be exactly the same (incl. column header meta data), irrespective of the io package version, because shapewrite invokes dbfwrite from the io package. IOW, the same dbfwrite will be invoked, directly or indirectly.

So, some more questions:

1. Exactly what are the differences in the .dbf tables that you see?

2. Did you enter the "which" command from an arbitrary directory, or from the directory you run your scripts from? Are you sure there is no dbfwrite.m or dbfread.m in your working directory?

3. What are the versions of the mapping and geometry packages that you have installed? Maybe those are somehow incompatible with io-2.6.0 in a way I didn't notice.
FYI I have mapping-1.4.0 and geometry 4.0.0.

IMO it is very bad idea to share your computer remotely to anyone you don't know well; I would never do that, even if I were desperate.

Philip Nienhuis <philipnienhuis>
Project Member
Wed 01 Apr 2020 03:50:02 PM UTC, comment #5: 

Dear Philip,
Really thanks. I am dealing with several issues I have to find a solution now in this confinement :)

Step by step:
Which dbfread and dbfwrite points to io folder. So, this is correct.

I have uninstalled octave and re-installed everything (octave and packages). I have been running the "test.m" that I attached. I gives bad results until I go down to io-2.4.11.

I have tested the results both QGIS and libreoffice.

The fault might lie with the Operative System given that you mention excel (I think you are using windows) and I am using kubuntu (ubuntu LTS 18.04).

For me it would not be a problem to share remotely my computer to anyone who wants to make any test.

Jaime Almonacid-Caballer <jaimepi>
Tue 31 Mar 2020 11:00:06 AM UTC, comment #4: 

Dear Jaime,

Thanks for uploading the example.

I cannot reproduce your issue.
I read the shapefile 'TestInput.shp' using shaperead.m (which invokes dbfread.m behind the scenes) and wrote it back as 'test_out.shp' using shapewrite.m (which invokes dbfwrite.m behind the scenes).
Next, I read both .dbf files Testinput.dbf and test_out.dbf into MS-Excel. That reads in separate invocations, so I copied the 'test_out' worksheet contents from Excel window #2 into a new worksheet next to the 'Testinput' worksheet. In a third, new sheet 'diff' I subtracted the values in sheet 'test_out' from those in 'Testinput'.
AFAICS the contents of both worksheets, and thus both .dbf-files, is the same.
That combined spreadsheet file is attached.
(I could also have done it in Octave itself using a.o., cell2mat, but I find it's easier to visually check & compare in a spreadsheet)

I also did this in LibreOffice; AFAICS only the column headers differ as regards meta info. But AFAIK that shouldn't matter.

Finally, after reading both datasets in QGIS I couldn't discern any differences either.

Could you have a stray dbfwrite.m or maybe dbfread.m somewhere in your path?
What does:
'which dbfwrite'
say? and what about:
'which dbfread'
(both should point to the same directory where the io package .m function files are living.)

Philip Nienhuis <philipnienhuis>
Project Member
Tue 31 Mar 2020 08:27:45 AM UTC, comment #3: 

Dears Mike and Markus.
Frist of all, my apologies for the way I published my doubt, I am not an expert in this environment.
I should have read with attention the news.
Finally, I have attached the information and simple code to test the problem. I am sure it is only due to dbfwrite.
The problem itself is that reading and writing the directly a dbf should lead to the same table and it does not.

Thanks again, I will try to do it better.

Jaime Almonacid-Caballer <jaimepi>
Mon 30 Mar 2020 04:52:27 PM UTC, comment #2: 

Please report one issue per bug report.

The xmlread function was deliberately removed from the io package, it's mentioned in the news for version 2.6.0, please see bug #58004 for more information.

Please give an example that shows the problem with dbfwrite for this bug report so that someone can try to reproduce the error and investigate.

Mike Miller <mtmiller>
Project Administrator
Mon 30 Mar 2020 04:51:48 PM UTC, comment #1: 

xmlread was (temporarily) removed from the io package. See the release notes:

Re-titling for the issue in the mapping package and CC'ing the package maintainer.

In the meantime, could you provide an example of what you were trying, what result you expected and what you got instead? Otherwise, I doubt anyone will be able to help.

Markus M├╝tzel <mmuetzel>
Project Member
Mon 30 Mar 2020 07:50:26 AM UTC, original submission:  

I did not know whether to mix both problems or not but they are linked to IO package.

Problem 1: xmlread does not appear in IO 2.6.0. even with "help xmlread"

Problem 2: Mapping package has the funcion shapewrite. It exports the attributes table (*.dbf) erroneously. I have tried to create this table using dbfwrite and results are also wrong. I do not know if it is due to a deeper interaction between packages.

SOLUTION (by the moment): going back to IO 2.4.13

Thanks a lot

PD: I use Octave 4.2.2. because it is the laste version apart from
 that released by flatpack.

Jaime Almonacid-Caballer <jaimepi>


(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:

Attached Files
file #48719:  test_Octave.zip added by jaimepi (69KiB - application/zip - The zipped folder containes an example of input/output and the code used to find the problem.)


Depends on the following items: None found

Items that depend on this one: None found


Carbon-Copy List
  • -email is unavailable- added by mmuetzel (Posted a comment)
  • -email is unavailable- added by mmuetzel
  • -email is unavailable- added by jaimepi (Submitted the item)
  • -email is unavailable- added by jaimepi

    Do you think this task is very important?
    If so, you can add your encouragement to it.
    This task has 0 encouragements so far.

    Only project members can vote.




    Follow 9 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2020-04-02 philipnienhuis StatusNeed Info => Invalid
        Open/ClosedOpen => Closed
    2020-04-01 philipnienhuis Release4.2.2 => other
    2020-03-31 jaimepi Attached File- => Added test_Octave.zip, #48719
    2020-03-30 mtmiller StatusNone => Need Info
        Summary[octave forge] (io) dbfwrite error => [octave forge] (io) dbfwrite / shapewrite error
    2020-03-30 mmuetzel Summarypackage IO 2.6.0 / xmlread missing / dbfwrite error (mapping pakage) => [octave forge] (io) dbfwrite error
        Carbon-Copy- => Added philipnienhuis
    2020-03-30 jaimepi Carbon-Copy- => Added jaialca@hotmail.com

    Back to the top

    Powered by Savane 3.5