bugGNU Octave - Bugs: bug #56809, Dots in script file names should...

 
 

bug #56809: Dots in script file names should be allowed

Submitter:  Michael Leitner <mleitner>
Submitted:  Mon 26 Aug 2019 01:25:41 PM UTC
   
 
Category:  Interpreter Severity:  3 - Normal
Priority:  5 - Normal Item Group:  Feature Request
Status:  Confirmed Assigned to:  None
Originator Name:  Open/Closed:  * Open
Release:  * dev Operating System:  * Any
Fixed Release:  None Planned Release:  None
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Mon 02 Sep 2019 01:45:14 AM UTC, comment #9: 

I agree with Mike.
A valid script name should be as the same as a function name or a variable name.
Because most punctuation characters are operators in Octave,
allowing punctuation characters in a script name is terrible!

Anonymous
Fri 30 Aug 2019 03:36:42 PM UTC, comment #8: 

I think I've made my point. In response to your latest,

> the concept of script files is recognized at an unnecessarily deep level in the present implementation


I don't think that's accurate. A script file is recognized at the evaluator level, after a parse tree has already been built from the input text.

I think that invalidates your last point, that a script name is known early enough that a decision could be made to parse the complete input line according to two different sets of rules.

Mike Miller <mtmiller>
Group Member
Fri 30 Aug 2019 12:54:39 PM UTC, comment #7: 

You see, I am not familiar with all peculiarities of octave's syntax, much less with the mechanisms how they are enforced. And if octave's syntax was drawn up from scratch (and no matlab precedent existed) and somebody asked me, I probably would vote against having the possibility of running scripts by just typing the script's filename excluding the suffix at the command line at all. After all, python and R shells are doing fine with the equivalents of source(...), which also in the present case would have no problems with dots, spaces or equalities.

It's just that on the one hand the possibility does exist and is arguably the standard way of running scripts used by people, and on the other hand it would be nice if dots were allowed in script file names (I don't care about spaces). And that it seemed to me that the concept of script files is recognized at an unnecessarily deep level in the present implementation (see the reported error for script(1) or script+1). It still seems to me that after splitting any entered command at the semicolons, a script name has always to appear in isolation. This would have allowed to distinguish the parsing into the first case "it is potentially a complex expression, but no script names appear" (including your case of "foo bar"), failing which the second case "it is a single script file name without any arguments or operands" would be tried, in which case the dot would not have a special meaning any more. Am I wrong here?


Michael Leitner <mleitner>
Fri 30 Aug 2019 12:24:23 PM UTC, comment #6: 

I agree with Mike.  I don't think we want to go down this path.

Script file resolution only happens at execution time, not when the code is parsed.

AFAIK, the only time we care about whether a symbol is defined as a variable is to determine whether to parse something like


ls -1


as a command, equivalent to the expression "ls ('-1')", or as a subtraction.  And that only happens at the command line, not inside scripts or functions.  And it happens when the expression is parsed, not when it is executed.

Even if you limit your proposal to the command line, you will have some trouble if you expect it to work for spaces.  Currently, the expression


foo bar


is equivalent to "foo ('bar')" if foo is not defined as a variable (again, only at the command line).  Now we would attempt to execute the script "foo bar.m" if the evaluation failed to find a function "foo"?

You are free to experiment with this idea and it may be possible to implement in some limited way but I don't think it is a good feature to add to Octave.

John W. Eaton <jwe>
Group administrator
Fri 30 Aug 2019 07:38:32 AM UTC, comment #5: 

Not exactly. In my view you do not have to search the string for a script file, you just take the whole string, append a ".m", and see if you find a corresponding script file. In this sense, the secondary grammar is trivial. And the primary grammar would be even simplified compared to the present case, because as it is, it seems to have a notion of which script files exist, and it even matches the tokens it finds against names of script files, even though this will only ever work if the string name encompasses the whole expression. In my proposal, you could drop the concept of script files completely from the primary grammar. And you do not have to care about spaces: issuing "my script" would fail in the primary grammar, but if there exists a file "my script.m", octave would happily execute it (the days of DOS' 8.3 file name restrictions are long gone, and spaces can be parts of file names in general, so why not in octave scripts?).

Why do you think that this is not a good idea, apart from the argument that is always valid, namely that there is an existing solution that works in most cases, while a potentially better solution would need human effort?

Michael Leitner <mleitner>
Thu 29 Aug 2019 09:45:44 PM UTC, comment #4: 

If I'm understanding you correctly, you are suggesting that scripts could be treated specially, since they can only be called by name with no arguments and cannot be part of larger expressions anyway. So essentially you are suggesting that if the normal grammar rules results in an expression that fails to evaluate, for some error or set of errors, and the expression contains no spaces, then a secondary grammar should be used instead, where the entire string is searched for a script file with a .m file extension. Is that about right?

I don't think that's a good idea.

Mike Miller <mtmiller>
Group Member
Thu 29 Aug 2019 08:43:46 PM UTC, comment #3: 

No, I wouldn't touch parsing. I see that it won't be possible to have arithmetic operators, dots and the like in function names, but script names don't need to be parsed. What I am proposing is that only if parsing errors out or if it finds that some tokens it has identified as operands are undefined, it is tried whether the whole command string  is an existing script file name. Thus, if it should happen that if you have a struct "script" defined that has a field "a", you wouldn't be able to call the script "script.a.m" by issuing the command "script.a", but in all other cases you would. You would even be able to name a script "a=b.m" and call it the normal way as long as b does not exist (I don't claim that this would be a good idea).

By the way, I have just found out that the interpreter (I don't know if this is the correct concept, what I mean is the stage of octave that takes the tokens returned by the parser and evaluates them) does have some notion of scripts: when you have script.m, calling


script(1)


returns "invalid use of script /tmp/script.m in index expression", while


script+1


returns "invalid call to script /tmp/script.m". I would have expected "'script' undefined near line ...", which is returned if script.m does not exist. Is there a reason for that? Can script names (that is, not as a string object delimited by quotes) in any sense be part of a regular command apart from making up the whole of the command? Couldn't thus the process be actually simplified by not having to check tokens against string names, and only if a parsing error occurs or some token is undefined, try to read the whole command as script name?

Michael Leitner <mleitner>
Thu 29 Aug 2019 07:00:43 PM UTC, comment #2: 

I'm not sure how this would work with the Octave syntax.

If the Octave parser sees an expression "script.a", that is first parsed as an identifier "script" followed by a subscript expression referencing a field named "a". That is before Octave knows whether "script" is a variable, function, script, class, etc.

I think it would be nice if Octave allowed functions and scripts to use hyphens, but unfortunately a script named "do-something" is parsed as a binary minus operation. These seem like the same to me. The names of script files must be valid identifiers. I'm not sure there's any way around that without making parsing dependent on context and evaluation, and in turn more complex and error prone.

Mike Miller <mtmiller>
Group Member
Thu 29 Aug 2019 02:33:57 PM UTC, comment #1: 

Confirmed.  I also became aware of this only recently (last month).  I would also like to have this feature, but have been relying on "source" for the time being.

Rik <rik5>
Group administrator
Mon 26 Aug 2019 01:25:41 PM UTC, original submission:  

If you have a script file called script.a.m, it is impossible to execute it in the normal way by just entering its name excluding the trailing .m (of course the syntax source("script.a.m") still works). Octave will complain that 'script' is undefined, obviously because it parses the command as a structure 'script', of which the field 'a' is requested. Interestingly, for a file script.1.m it doesn't even get so far, seemingly because field names cannot start with a digit, thus giving you a parsing error.

So the dot character has a special meaning in the syntax. Obviously Octave can now go the easy way and just forbid using it more than exactly once (before the suffix "m") in script file names. In this case, this at least borders onto a documentation bug, because in the documentation the only thing I find about script file names is under 11.10 Script Files "Octave uses the same rules to search for script files as it does to search for function files.", while 11.9 Function Files does not say anything, and only 11.2 Defining Functions says "A valid function name is like a valid variable name: a sequence of letters, digits and underscores, not starting with a digit."

However, I would argue that this restriction is unnecessary: if parsing errors out because the dot is followed by a digit, you can still try to interpret this as a script file name, the same applies if you find the leading string undefined. This does not hurt performance, as it on the one hand happens after parsing has been tried, and on the other hand also such script file names won't be much slower than those without dot, as I am quite sure that searching for a script file in the load path is already much slower than trying to parse an expression. Further, I would find it beneficial to be able to naturally name scripts for instance "integrate_sin_step0.1_midpoint.m", which comes natural if you use Octave as numerical environment for doing simulations (a student just came to me with such an issue -- it was only five minutes for me, but she probably had invested more time before asking me). I see no potential regressions with such an approach. Of course, I cannot judge the effort such a change would entail, necessitating to catch the error before it is issued, try to interpret it as a script file name, and if that should fail, not issue the corresponding error, but the original one.

Michael Leitner <mleitner>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by jwe (Posted a comment)
  • -email is unavailable- added by rik5 (Posted a comment)
  • -email is unavailable- added by mleitner (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only group members can vote.

     

    Follow 2 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2019-08-29 rik5 StatusNone Confirmed
        SummaryDots in script file names Dots in script file names should be allowed

    Back to the top

    Powered by Savane 3.13-758e.
    Corresponding source code