bugGNU Octave - Bugs: bug #62417, 'format' errors on names with...

 
 

bug #62417: 'format' errors on names with spaces when calling with function form

Submitter:  Georg Wiora <gwiora>
Submitted:  Thu 05 May 2022 01:53:58 PM UTC
   
 
Category:  Octave Function Severity:  2 - Minor
Priority:  3 - Low Item Group:  Unexpected Error or Warning
Status:  Fixed Assigned to:  None
Originator Name:  gwiora Open/Closed:  * Closed
Release:  * 7.1.0 Operating System:  * Any
Fixed Release:  None Planned Release:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Sun 22 May 2022 05:39:39 PM UTC, comment #26: 

I like Rik's proposal in comment #24. I pushed a change to the stable branch that will hopefully avoid that the formatting of the docstring in the terminal causes confusion:
https://hg.savannah.gnu.org/hgweb/octave/rev/4d74446020ad

Marking as ready for test.

Markus Mützel <mmuetzel>
Group administrator
Mon 16 May 2022 05:55:26 PM UTC, comment #25: 

is there a historical reason we publish the format name as 'short e' instead of 'shorte'? (it's been that way at least back to v3, which is as far as I can google old docs).

'shorte' and ('short','e') are both needed for full compatability, even if the latter appears undocumented on matlab's side. while i think modification to accept 'short e' wouldn't be terribly complex, just removing it from the docs would also eliminate the ambiguity.

Nicholas Jankowski <nrjank>
Group Member
Wed 11 May 2022 05:45:21 PM UTC, comment #24: 

I think we are using Texinfo tags appropriately.  For reference,


@code : for code (mono-spaced font).  In Info format the block is enclosed in single quotes.
@command : for UNIX commands, such as @command{ls}
@env  : for environment variables, such as @env{HOME}
@file : file names, such as @file{~/.hgrc}
@kbd  : keyboard input typed by user
@key  : a key, such as @key{RET} for Return
@option : for options, such as @option{-l}
@samp : catch-all for any text that should be set off from the surrounding documentation, but is not one of the categories above.  The text will be enclosed by single quotes in Info format.


The @qcode macro is one I designed because the single quotes around a piece of text that is already quoted look horrible in Info format.


@code{"Text"} -> '"Text"' in Info
@qcode{"Text"} -> "Text" in Info


I also created the @tcode macro for code that ends with a transpose operator (').


@code{y = x'} -> 'y = x''  in Info
@tcode{y = x'} -> y = x'   in Info


Perhaps we shouldn't bother to document "short e" form and just document "shorte".  Both will be accepted by the format command, but the second form doesn't have documentation problems.


Rik <rik5>
Group administrator
Wed 11 May 2022 03:18:49 PM UTC, comment #23: 

Reposting my previous comment because of a botched verbatim tag...

I understand that the quoting caused confusion, but I think this is really a problem due to a mixture of bad Texinfo markup in the source file.  In the docstring for the format command in pr-output.cc, we have four tables.  Three use "@table @code" like this:


@table @code
@item short
...
@item long
...
@item  short e
@itemx long e
...
@end table


which results in the items being typeset in typewriter font in PDF and similar output formats and surrounding in single quote characters in the Info and the output displayed by the command-line help function.

The other table uses "@table @asis" with the @item entries sometimes tagged with @qcode{}, like this:


@table @asis
@item  @qcode{"+"}
@itemx @qcode{"+"} @qcode{"@var{chars}"}
@itemx @code{plus}
@itemx @code{plus @var{chars}}
...
@item bank
...
@item native-hex
...
@end table


To further confuse things, we also have


@item  short eng
@itemx long eng
Identical to @samp{short e} or @samp{long e} but displays the value using an
engineering format, where the exponent is divisible by 3.  For example, with
the @samp{short eng} format, @code{10 * pi} is displayed as @code{31.416e+00}.


By using @samp, you get a typewriter font surrounded by quotation marks in typeset output and text enclosed in single quote characters in ASCII output.  The use of @samp might just be wrong.  I don't remember the guidelines about what tags to use for what purpose.

Part of the problem is that in the Info format and what is displayed by the help function the convention is to use single quotes to display @code elements because simple ASCII output doesn't have a typewriter font.

I'm not sure what is best.  Maybe the guidelines for Texinfo formatting are not being followed correctly or they are just outdated.  The quotes in simple ASCII output were probably fine in 1990 but easily misinterpreted today.  Maybe we should try to improve the help function so that it displays nicely typeset output when possible?  That might avoid confusion in about 99% of all use cases in these modern times.

John W. Eaton <jwe>
Group administrator
Wed 11 May 2022 03:17:10 PM UTC, comment #22: 

I understand that the quoting caused confusion, but I think this is really a problem due to a mixture of bad Texinfo markup in the source file.  In the docstring for the format command in pr-output.cc, we have four tables.  Three use "@table @code" like this:


@table @code
@item short
...
@item long
...
@item  short e
@itemx long e
...
@end table


which results in the items being typeset in typewriter font in PDF and similar output formats and surrounding in single quote characters in the Info and the output displayed by the command-line help function.

The other table uses "@table @asis" with the @item entries sometimes tagged with @qcode{}, like this:


@table @asis
@item  @qcode{"+"}
@itemx @qcode{"+"} @qcode{"@var{chars}"}
@itemx @code{plus}
@itemx @code{plus @var{chars}}
...
@item bank
...
@item native-hex
...
@end table
+verbatim+

To further confuse things, we also have

+verbatim+
@item  short eng
@itemx long eng
Identical to @samp{short e} or @samp{long e} but displays the value using an
engineering format, where the exponent is divisible by 3.  For example, with
the @samp{short eng} format, @code{10 * pi} is displayed as @code{31.416e+00}.


By using @samp, you get a typewriter font surrounded by quotation marks in typeset output and text enclosed in single quote characters in ASCII output.  The use of @samp might just be wrong.  I don't remember the guidelines about what tags to use for what purpose.

Part of the problem is that in the Info format and what is displayed by the help function the convention is to use single quotes to display @code elements because simple ASCII output doesn't have a typewriter font.

I'm not sure what is best.  Maybe the guidelines for Texinfo formatting are not being followed correctly or they are just outdated.  The quotes in simple ASCII output were probably fine in 1990 but easily misinterpreted today.  Maybe we should try to improve the help function so that it displays nicely typeset output when possible?  That might avoid confusion in about 99% of all use cases in these modern times.

John W. Eaton <jwe>
Group administrator
Wed 11 May 2022 02:44:25 PM UTC, comment #21: 

re: comment 18:  


works:
format short e
format shorte
format('short','e')
format('shorte')

errors:
format "short e"
format("short e")


all of which is fine, except this isn't a matter of combining different option strings, as in 'copyfile a b'.  we explicitly name the format option as 'short e', with quotes in many places.  e, E, g, and Eng are not independent options. they only exist following long or short', and those are listed as different specific option strings:


     'long'
          Fixed point format with 16 significant figures.

          As with the 'short' format, Octave will switch to an
          exponential 'e' format if it is unable to format a matrix
          properly using the current format.

     'short e'
     'long e'
          Exponential format.  The number to be represented is split
          between a mantissa and an exponent (power of 10).  The
          mantissa has 5 significant digits in the short format.  In the
          long format, double values are displayed with 16 significant
          digits and single values are displayed with 8.  For example,
          with the 'short e' format, 'pi' is displayed as '3.1416e+00'.

     'short g'
     'long g'
          Optimally choose between fixed point and exponential format
          based on the magnitude of the number.  For example, with the
          'short g' format, 'pi .^ [2; 4; 8; 16; 32]' is displayed as

               ans =

                     9.8696
                     97.409
                     9488.5
                 9.0032e+07
                 8.1058e+15

     'short eng'
     'long eng'
          Identical to 'short e' or 'long e' but displays the value
          using an engineering format, where the exponent is divisible
          by 3.  For example, with the 'short eng' format, '10 * pi' is
          displayed as '31.416e+00'.
...


it would be like if in integral we described the option 'Waypoints' as 'Way points', but didn't tell the user they can't call it as 'Way points'.  I had to look at the code to realize why the 'short eng' format can't be called as 'short eng'.

in m-code it would be simple to use a switch/case with case {'shorteng, 'short eng'} to capture both, without any ambiguity in intent, and to match the documentation as published. I still support either a doc change to clarify the conflicting messaging, or doing the equivalent in the compiled function.

Nicholas Jankowski <nrjank>
Group Member
Wed 11 May 2022 07:13:16 AM UTC, comment #20: 

Thanky Nicholas, your Comment #17 correctly states the original cause of my confusion. The arguments are quoted including the space wherever mentioned in running text. If found this really misleading and it would be easy to repair.

Georg Wiora <gwiora>
Wed 11 May 2022 06:36:12 AM UTC, comment #19: 

To clarify what I meant with the but about multiple string arguments: e.g., `copyfile a b` is not the same as `copyfile ("a b")`. We shouldn't need to describe that fact for all functions that take multiple string arguments.

Markus Mützel <mmuetzel>
Group administrator
Wed 11 May 2022 05:46:29 AM UTC, comment #18: 

I don't have access to Octave currently. Did you try the command syntax form `format "short e"`? Does that work?
If it doesn't, I don't think we should document that spaces in the format arguments are not allowed only in function syntax...

Markus Mützel <mmuetzel>
Group administrator
Wed 11 May 2022 05:11:01 AM UTC, comment #17: 

I think the main point or confusion comes from exactly what was stated in comment #0. The documentation explicitly lists those format names as two word names.

'short e'
'short eng'
'long e'
'long eng'
'short g'
'long g'

The documentation even renders them in a single set of quotations in some contexts (at least with help format) by using @code for the list, making it appear a single string with the space as a character in it.  In almost every other function, a specified option name is a single string entered to activate said option. This is the only place I've seen where that isn't the case. 

Now, this is only an issue with the function calling form of format, which until this report I had never considered using, and that calling method wasn't even documented until today. But now that it is a published method, it seems that "an error will occur if you call the option the way it appears we've told you to do so" should be documented if the behavior is to be kept.

Nicholas Jankowski <nrjank>
Group Member
Wed 11 May 2022 04:31:10 AM UTC, comment #16: 

That's what I figured. So, the function doesn't take spaces in its arguments (no matter whether it is called in function or command style - see e.g. `format "short e"`). To avoid possible confusion, get rid of the first part of the proposed documentation change imho.
While the end of the proposed documentation change is technically correct, I don't think this is the correct place to document that. We would need to document that fact for all functions that take multiple string arguments. Instead of repeating the same thing for all of those functions, we should document that in a more central location (if it isn't already).
This leaves us with the fact that it isn't documented that the function also allows contracting the "e" to some options. We should document just that imho.

Markus Mützel <mmuetzel>
Group administrator
Wed 11 May 2022 04:00:52 AM UTC, comment #15: 

Nothing indicates specifically that a command style function call has happened.  The parser simply converts things that look like


format short e


to the equivalent index expression


format ('short', 'e')


Then the usual argument processing and dispatching happens when that expression is evaluated.

John W. Eaton <jwe>
Group administrator
Tue 10 May 2022 10:58:27 PM UTC, comment #14: 

by my understanding of the code, not so much intentional checking as an artifact of how it processes the inputs.  'format short e' has two separate options, 'short' and 'e', passed to it. if the code sees 'short' it checks to see if the next input is 'e' or another valid option, and if so processes it as 'shorte', etc.  for function input, format('short e') has only one option input, and none match 'short e', but format('shorte') matches, and format('short','e') behaves the same as 'format short e'

Nicholas Jankowski <nrjank>
Group Member
Tue 10 May 2022 10:42:49 PM UTC, comment #13: 

I would be surprised if that documentation change is correct. Does the function really check if it is called in function or command syntax? Is that even possible?

Markus Mützel <mmuetzel>
Group administrator
Tue 10 May 2022 08:33:45 PM UTC, comment #12: 

apparently a sloppy combination of the preview and back button use has the wrong text in comment #11. patch text in file #53185 is:


When using the functional form of @code{format}, option strings cannot contain
spaces.  Format options with spaces must be specified with spaces removed or
as multiple function inputs, for example, as @code{format("shorte")} or
@code{format("short", "e")}.  Specifying an option with spaces, e.g.,
@code{format("short e")} will produce an error.


Nicholas Jankowski <nrjank>
Group Member
Tue 10 May 2022 08:26:23 PM UTC, comment #11: 

I think a doc change makes sense, since we go to the trouble of specifying all of our formats with spaces.  Attached patch adds the following toward the end of the format docstring:


When using the functional form of @code{format}, option strings cannot contain
spaces.  Format options with spaces must be specified with spaces removed or
as multiple inputs, for example, as @code{format("shorte")} or
@code{format("short", "e")}.


bug #62431 created to capture the 'a = format' item.

(file #53185)

Nicholas Jankowski <nrjank>
Group Member
Tue 10 May 2022 07:31:26 PM UTC, comment #10: 

I think this is just confusion about command form versus function form and that there is no need to change to try and support


format ('short e')


If someone wants to make a documentation change to clarify this that might be okay.  Otherwise, I would just close this as "Won't Fix" and move on.

Rik <rik5>
Group administrator
Tue 10 May 2022 07:18:19 PM UTC, comment #9: 

I don't see any need to transform the argument in this way.

If Matlab produces a classdef object for "a = format" now, then I suppose we should do the same if someone wants to work on that.  Returning a struct might be better than no change, but if we are doing some work here, why not just return a compatible object?


John W. Eaton <jwe>
Group administrator
Tue 10 May 2022 06:32:02 PM UTC, comment #8: 

is the desired path forward just to document that function form calls must not include spaces, or how hard would it be to replace the tolower step with the c++ equivalet of:


arg = arg(!isspace(tolower(arg)))



Nicholas Jankowski <nrjank>
Group Member
Tue 10 May 2022 06:24:43 PM UTC, comment #7: 

renamed bug as patch for bugs #62430 addresses the undocumented function form concern.

Nicholas Jankowski <nrjank>
Group Member
Mon 09 May 2022 05:22:42 PM UTC, comment #6: 

correct, matlab errors on format('short e'):


>> format shorte
>> format short e
>> format('short','e')
>> format('short e')
Error using format
Invalid format option short e.

except for the latter, the above all make the same format change to:


a = format

a =

  DisplayFormatOptions with properties:

    NumericFormat: "shortE"
      LineSpacing: "loose"


so, all their documentation list as single words, but they accept two-word format states except for the function call.

All of Octave's documentation list the format states as two words. There is no mention of a 'shorte' option, just 'short e'.  Since Short and E are separate format specifiers, this does make some sense, and needing to send them to the function form as two separate strings makes sense in that context. 

Maybe then this is primarily a documentation problem.

Nicholas Jankowski <nrjank>
Group Member
Mon 09 May 2022 05:04:01 PM UTC, comment #5: 

By the rules of command-style function invocation These are equivalent:


format short e
format ('short', 'e')


There is no 'short e' option.  Does Matlab accept


format ('short e')


?

John W. Eaton <jwe>
Group administrator
Mon 09 May 2022 04:39:23 PM UTC, comment #4: 

renamed reprot to, i believe, better capture the state of the bug.


>> format short e
>> format shorte
>> format ('shorte')
>> format ('short e')
error: format: unrecognized format state 'short e'


Nicholas Jankowski <nrjank>
Group Member
Mon 09 May 2022 04:36:26 PM UTC, comment #3: 

I also notice that our documentation does not specify a function form for format at all. Is that intentional?

it also seems matlab recently introduced a 'default' format to be an explicit equivalent of format without any arguments, which currently causes an 'unrecognized format' error.  (added as bug #62430)

Nicholas Jankowski <nrjank>
Group Member
Mon 09 May 2022 04:11:16 PM UTC, comment #2: 

It was maybe my mistake to use quotes on the command line.
The following is not accepted:

>> format "short e"
error: format: unrecognized format state 'short e'

while these two command forms are accepted:

>> format short e
>> format "shorte"


But even when you use the function form

>> format("shorte")
>> format("short e")
error: format: unrecognized format state 'short e'

only the first version is accepted.

I have to admit that I am a bit puzzled by these results... Maybe it is more than a documentation bug?

Georg Wiora <gwiora>
Mon 09 May 2022 03:56:07 PM UTC, comment #1: 

What do you think is wrong?  "short e" and "shorte" are both accepted forms for the same format and need to be accepted for Matlab compatibility (https://www.mathworks.com/help/matlab/ref/format.html).

Rik <rik5>
Group administrator
Thu 05 May 2022 01:53:58 PM UTC, original submission:  

The names of the numeric formats than can be specified as argument to the "format" command seem to be misspelled on the doc page.

The documentation lists the following format names:

  • "short e"
  • "short eng"
  • "long e"
  • "long eng"
  • "short g"
  • "long g"


I cannot exclude that the implementation of "format" is wrong. At least it is accepting the following formats in the current version 7.1.0

  • "shorte"
  • "shorteng"
  • "longe"
  • "longeng"
  • "shortg"
  • "longg"



Georg Wiora <gwiora>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #53185:  format_doc_clarifyspaces_bug62417.diff added by nrjank (1KiB - application/octet-stream)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by mmuetzel (Posted a comment)
  • -email is unavailable- added by jwe (Posted a comment)
  • -email is unavailable- added by nrjank (Posted a comment)
  • -email is unavailable- added by rik5 (Posted a comment)
  • -email is unavailable- added by gwiora (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only group members can vote.

     

    Follow 13 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2022-05-24 rik5 StatusReady For Test Fixed
        Open/ClosedOpen Closed
    2022-05-22 mmuetzel StatusPatch Submitted Ready For Test
    2022-05-10 nrjank Attached File- Added format_doc_clarifyspaces_bug62417.diff, #53185
        StatusConfirmed Patch Submitted
    2022-05-10 rik5 Priority5 - Normal 3 - Low
    2022-05-10 nrjank SummaryFunction form of 'format' undocumented and errors on format names with spaces 'format' errors on names with spaces when calling with function form
    2022-05-09 nrjank CategoryDocumentation Octave Function
        Severity3 - Normal 2 - Minor
    2022-05-09 nrjank Item GroupDocumentation Unexpected Error or Warning
        StatusNeed Info Confirmed
        SummaryFormat names misspelled in documentation of format() Function form of 'format' undocumented and errors on format names with spaces
    2022-05-09 rik5 StatusNone Need Info

    Back to the top

    Powered by Savane 3.13-d3ae.
    Corresponding source code