Sun 23 Apr 2017 11:31:40 AM UTC, comment #11:
If no one beats me to it, I'll have a look in the coming weeks.
I think the delimiter assessment code, currently acting on the header lines if these are present, ultimately should be based on the purely numeric section.
This implies some fseek-ing hence and forth, but I suppose the required execution time for that is peanuts compared to reading (large amounts of) numeric data further on.
I also have a basic idea on how to deal with column headers comprising units enclosed in parens or brackets separated by whitespace, once the delimiter assessment works reliably. But I won't try AI here :-), just basic guesswork and if that fails just return cellstr.
(Just wondering what to do if row headers are also present...)
|
Tue 18 Apr 2017 08:04:26 PM UTC, comment #10:
@JWE:
AFAICT importdata() invokes dlmread once it knows the delimiter so it shouldn't be that slow.
IMO user-convenience functions like importdata.m quickly tend to turn into dinosaurs with a lot of built-in guesswork (or if you really want, AI :-) ).
I think attempts to mimick this guesswork/AI is a nice job for a real hobbyist; maybe we shouldn't pursue this too much.
Looking at Octave's code I see a lot of opportunities for misinterpretation of the header line. E.g., when one of more of the column headers comprises just digits.
The line of hyphens is skipped because it is fed to str2double that returns a NaN (IOW, not numeric data) and then the line is added to the headerlines collection.
|
Tue 18 Apr 2017 07:48:33 PM UTC, comment #9:
... and it also fails on Linux =>
OS => Any
|
Tue 18 Apr 2017 07:47:40 PM UTC, comment #8:
Hmmm, the "bug" I mentioned refers of course to the failing regexp in L.174.
|
Tue 18 Apr 2017 07:46:07 PM UTC, comment #7:
@OP:
You can at least recover the numeric data using:
i.e., specify the delimiter explicitly.
A quick analysis shows that the regexp in L.174 (in subfunc importdata_asci) turns up "g" rather than a space as delimiter.
But even when specifying the delimiter explicitly the header isn't parsed correctly, so there's a bug I think.
That Matlab seems a bit smarter than Octave as regards parsing the header is a mere compatibility issue.
Once you got the data read fine, the header entries can be parsed using some magic provided you know what the header is expected to look like. In this specific case:
I also leave this open as I have other priorities the next weeks.
Status => confirmed
Release => dev
|
Tue 18 Apr 2017 07:45:08 PM UTC, comment #6:
Yes, please leave this open.
How does Matlab recognize column headers when they contain spaces and the field separator is also spaces? Does it do some paren matching? There must be some kind of fuzzy matching/guessing that it is doing.
Also, how does it know to skip the line of "-" characters? Again, some kind of guesswork? A human can easily tell what the header is. Maybe Matlab looks for the first line that has something that looks like a row of numbers? I don't know.
I downloaded the file and it appears to be 9+MB. I truncated it to 10 lines of data and uploaded it. It should be attached to this comment. The abbreviated file still shows the same problem, so we don't really need all the extra lines for this report.
OTOH, I also noticed that importdata is quite slow for large files of only numbers (I removed the header lines to see what it would do with this file). But that should probably be a separate report.
(file #40439)
|
Tue 18 Apr 2017 05:59:44 PM UTC, comment #5:
Ok, maybe, I don't know for certain if the file size is the issue or not, I usually only deal with attachments that are kB rather than MB.
Regardless, I will leave this open for someone who knows more about importdata to ponder. I don't use the function or know how it is intended to work, but this seems like a Matlab compatibility issue that someone may want to address.
|
Tue 18 Apr 2017 05:54:37 PM UTC, comment #4:
Ok, I thought the file-size was ok, because windows reports 9.673 KB which is under the limit which is shown.
As the file is an export from CST Microwave Studio (a tool we use often), changing the header everytime is not an option for.
Although the behavior differs from Matlab, as mentioned before, I have a fix for our use in place. So, I'm ok with closing this bug.
|
Tue 18 Apr 2017 05:08:44 PM UTC, comment #3:
You should be able to attach the file if you compress it, there is a file size limitation.
Anyway, I can confirm, and changing the file header from this
to this
fixes it for me. In other words eliminating the extra labels in square brackets in the first row.
|
Tue 18 Apr 2017 03:11:42 PM UTC, comment #2:
I won't upload, I get the following error:
No group chosen: nogroup.
Please find the file here: http://www.dirix.nu/Plot5G.txt
I have further found that in this case "textscan("Plot5G.txt',' ',2) (E.g. space-separated, and 2 headerlines) at least parses the file.
|
Tue 18 Apr 2017 03:01:52 PM UTC, comment #1:
I cannot reproduce your problem with a simple .txt file as follows:
A B
1 2
3 4
saved as test.txt
Matlab r2016b:
Octave 4.2.0 and 4.3.0+ (the latter dev. version):
If you want us to look further into it please upload a test file that works in Matlab and provokes the bug in Octave.
Status => Need info
|
Sun 16 Apr 2017 12:57:02 PM UTC, original submission:
In Octave, using importdata directly on a file return a cell containing strings for each line of the file.
However in Matlab the same command returns a struct containing separate textdata cell and data maxtrix.
Matlab output:
AData = importdata('Plot5G.txt')
AData =
data: [65160x8 double]
textdata: {2x1 cell}
Octave:
AData = importdata('Plot5G.txt')
AData =
{
[1,1] = Theta [deg.] Phi [deg.] Abs(Grlz)[dB ] Abs(Cross)[dB ] Phase(Cros
s)[deg.] Abs(Copol)[dB ] Phase(Copol)[deg.] Ax.Ratio[dB ]
[2,1] = ------------------------------------------------------------------------------
------------------------------------------------------------------------
[3,1] = 0.000 0.000 1.322e+01 -1.560e-01
194.336 1.302e+01 62.815 1.588e+01
[4,1] = 1.000 0.000 1.330e+01 2.045e-01
186.123 1.309e+01 54.157 1.566e+01
[5,1] = 2.000 0.000 1.338e+01 5.023e-01
178.192 1.315e+01 45.538 1.554e+01
.....
|