bugGNU Octave - Bugs: bug #42883, Various issues relating to the...

 
 

bug #42883: Various issues relating to the lack of re-compilation of .m files changed during script execution

Submitted by:  Jonathan Dodd <gnathan87>
Submitted on:  Thu 31 Jul 2014 02:34:46 AM UTC  
 
Category: InterpreterSeverity: 3 - Normal
Priority: 5 - NormalItem Group: Other
Status: Need InfoAssigned to: None
Originator Name: Jonathan DoddOpen/Closed: Open
Release: 3.8.0Operating System: GNU/Linux

Add a New Comment (Rich MarkupRich Markup):
   

You are not logged in

Please log in, so followups can be emailed to you.

 

Thu 31 Jul 2014 11:02:01 PM UTC, comment #4:

I don't think run and source have to behave identically. If the filename given to run has a directory component, it calls cd. I think that should cause the load path to be reexamined, so could affect timestamp checking. Source doesn't perform the cd, so it wouldn't affect the load path.

The timestamps should be checked at each prompt. I think that's why you see the difference in behavior with a script that is running normally vs. in debug mode, because in debug mode, you are prompted at each breakpoint or while stepping through a function. Perhaps the easiest thing to do would be to not update the value of Vlast_prompt_time when displaying the debug prompt.

I wonder whether the "less than 1 second ago" behavior is some arbitrary limit on the frequency of time stamp checking in Matlab, or if it is a limitation of the resolution of the file time stamps (or the way that Matlab is checking them)?

John W. Eaton <jwe>
Project Administrator
Thu 31 Jul 2014 08:34:00 PM UTC, comment #3:

Mike - As to the race condition in Matlab: yes. As to what I am suggesting: this really depends on whether it is considered important to follow Matlab or not. Even if not, I would still raise the issue of the difference in behaviour between normal execution and debug mode. If the file is not resourced during normal execution then neither should it be resourced while debugging.

By the way, as I am reading it, Carne suggests below that lack of resourcing is also seen on the prompt when you just type the filename. In accordance with the documentation (which claims that file timestamps are checked at each new prompt), I do not get this behaviour.

Also just to note that run() calls source(), so at least insofar as re-sourcing is concerned, these should work identically.

Anonymous
Thu 31 Jul 2014 04:27:35 PM UTC, comment #2:

I'll just comment on this since I helped the OP debug this on IRC. I'm not sure if this is a bug. Matlab's behaviour is weird and possibly not worth replicating, but we're behaving differently so a discussion may be worth.

In short, the possible issues are:

  • entering filename at the prompt behaves different than Matlab. Matlab reads the file if it has changed less than 1 second ago, Octave never does;
  • using run() in Octave will always re-source the file while Matlab behaves the same as naming file, i.e., re-source if changed less than 1 second ago;
  • Octave documentation suggests that run(), source(), and naming file at the prompt are the same but the rules for re-sourcing the file are different.

With examples:

If you have a m file in the current directory with 'x = 5', and you just enter the filename at the prompt, you get x on the environment. But if you then change the file for 'x = 6' and do it again, it will ignore the changes to the file as can be seen by calling testbug(20):

Matlab behaviour seems to check the timestamp of the file (with 1 second resolution), and re-source the file if it is different. This 1 second time resolution can lead to weird bugs where for the first 7 iterations prints 1, then prints 8 for a while, and then 17 until the end.

The behaviour is also different if run() or source() are used. In Octave it will always re-source the file but in Matlab, run() behaves exactly like just naming the file at the prompt (note that Matlab does not have source() ).

If we decide against following Matlab's behaviour here, then there's only the documentation bug of source() and run() which suggests that their usage is the same as typing the filename at the prompt.

Carnë Draug <carandraug>
Project Member
Thu 31 Jul 2014 01:24:49 PM UTC, comment #1:

Thanks for your bug report. So to clarify, you are suggesting that the Matlab behavior is undesirable in this case because it seems to be a race condition? And you are suggesting the Octave documentation be updated to explicitly state that m-files will not be re-read if they are modified while a long-running script is executing? Or are you suggesting any changes in behavior as well?

Mike Miller <mtmiller>
Project Member
Thu 31 Jul 2014 02:34:46 AM UTC, original submission:

This report relates to the lack of re-compilation of a .m file when it changes during the execution of a script. This may be intentional/desirable for performance reasons. However, it seems to differ from Matlab, causes a discrepancy between normal execution and execution in debug mode, and should be made explicit in the documentation.

The context in which I came across this behaviour related to an existing non-Octave program adapted to output matrices in .m format. A loop in my Octave script ran this program with different input parameters before calling the resulting .m file to load the data. I had expected each new data set to be read from the output file in turn, but in fact only the first data set was ever used.
Curiously, each new data set was correctly loaded in debug mode. This may be because Octave checks for changes at each new line when running interactively, as explained here. Thus, a changing .m file is not re-compiled when a script is running normally, but is re-compiled when step-debugging - clearly undesirable.

While the documentation does cover the re-compilation of changing files, it is not immediately obvious that files will not be recompiled during the execution of a script - even with ignore_function_time_stamp("none"). I therefore suggest that this is made explicit, and it is explained that if .m files change during script execution, the relevant function should should either be cleared, or the script file run using source() or run(), which ensure re-compilation.
Finally, in contrast to Octave, Matlab does seem to re-compile functions during script execution, but only after a time delay of about 1 second. However, experiments seemed to indicate that this behavior was in fact quite undesirable, producing different outputs depending on the time at which the script was executed (NB this was relayed to me on IRC - I am unsure whether it related to straight calls, calls through run() or both).

Example

See attachment. Comment out lines in foo() as appropriate. using run() or source() will result in y=1, 2, ... , 10. Calling testb directly results in y = 1, 1, ... , 1. Matlab may produce something like y = 1, 1, 1, 1, 6, 6, 6, 6, 6, 6, depending on the run.

Jonathan Dodd <gnathan87>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach File(s):
   
   
Comment:
   

Attached Files
file #31804:  recompiletest.m added by gnathan87 (274B - text/x-objcsrc)

 

Depends on the following items: None found

Digest:
   bug dependencies.

 

Carbon-Copy List
  • -unavailable- added by jwe (Posted a comment)
  • -unavailable- added by mtmiller (Posted a comment)
  • -unavailable- added by gnathan87 (Submitted the item)
  •  

    Do you think this task is very important?
    If so, you can click here to add your encouragement to it.
    This task has 0 encouragements so far.

    Only project members can vote.

     

    Please enter the title of George Orwell's famous dystopian book (it's a date):

     

     

    Follow 3 latest changes.

    Date Changed By Updated Field Previous Value => Replaced By
    Thu 31 Jul 2014 01:24:49 PM UTCmtmillerStatusNone=>Need Info
    Thu 31 Jul 2014 01:20:11 PM UTCmtmillerDependencies-=>bugs #42882 is dependent
    Thu 31 Jul 2014 02:34:46 AM UTCgnathan87Attached File-=>Added recompiletest.m, #31804

    Back to the top


    Powered by Savane 3.1-cleanup