bugGNU Octave - Bugs: bug #52556, Octave stops writing to console on...

 
 

bug #52556: Octave stops writing to console on Windows 7

Submitted by:  Georg Wiora <gwiora>
Submitted on:  Thu 30 Nov 2017 08:58:19 AM UTC  
 
Category:  GUI Severity:  3 - Normal
Priority:  2 Item Group:  None
Status:  None Assigned to:  None
Originator Name:  gwiora Open/Closed:  Open
Release:  dev Operating System:  Microsoft Windows

Add a New Comment (Rich Markup)
   

You are not logged in

Please log in, so followups can be emailed to you.

 

( Jump to the original submission)

Thu 17 Jan 2019 02:35:21 PM UTC, comment #15: 

Windows 7 is in its extended support period and won't get any updates in less than a year from now.
I also don't have access to any Windows 7 machines to test myself.
Considering these and the fact that jwe was considering to replace the command window with a qt text widget anyway, I have little motivation to code around the quirks of Windows 7's old command prompt.
Nonetheless, if someone else wants to work on a fix, I am willing to review.

Markus Mützel <mmuetzel>
Project Member
Tue 11 Dec 2018 07:53:58 AM UTC, comment #14: 

@Comment #12: I am using Windows 7.

Georg Wiora <gwiora>
Mon 10 Dec 2018 06:25:57 PM UTC, comment #13: 

(Philip here)
I vaguely remember something about garbled characters in the console but those memories need a bit refreshing.

Maybe related, but I do regularly see that there's no more output to the GUI terminal after what looks like some buffer overflow that seems to wipe code for displaying text in the terminal.
E.g., read a big text file with:

more off;
fid = fopen (<some filename of a fairly big file>);
txt = fread (fid, Inf, "*char")
## note absent semicolon at end of last line

... and then very often there's no more output shown at all.
Doing:

more on

then and trying typing commands indicates that the pager line is displayed but with background color (i.e., invisible).
I couldn't yet establish a robust test case so I didn't report it yet, but it could be relevant here (just guessing).

Philip Nienhuis <philipnienhuis>
Project Member
Mon 10 Dec 2018 05:07:19 PM UTC, comment #12: 

@Georg: Your test works correctly for me in Octave 4.4.1. I see:

>> help function_with_umlaut2

'function_with_umlaut2' is a function from the file D:\SVN\Octave\function_with_umlaut2.m

function Test_UTF_8() ÄÖÜäöüß This text is printed on console

Additional help for built-in functions and operators is
available in the online version of the manual.  Use the command
'doc <topic>' to search the manual index.

Help and information about Octave is also available on the WWW
at https://www.octave.org and via the help@octave.org
mailing list.

Also subsequent calls of fprintf still work.

But for 4.3.0+, the console output of "help sym" contains odd characters if the mfile_encoding is set to SYSTEM. This is a regression.
I'll open a new bug report for that.

Which version of Windows are you using? If I remember correctly, we (Philip?) saw that the Windows console prior to Windows 10 was buggy when it comes to UTF-8 (even worse in Windows XP).

Markus Mützel <mmuetzel>
Project Member
Mon 10 Dec 2018 08:44:01 AM UTC, comment #11: 

All my files seem to be UTF-8 encoded. Octave gui editor is showing this encoding and octave gui preferences are also set to UTF-8. I even tried to change the file to using UTF-8 with BOM. and added the @documentencoding tag to the header:

## -*- texinfo -*-
## @documentencoding UTF-8
## function Test_UTF_8()
## ÄÖÜäöüß
## This text is printed on console

But Octave still stops printing after end of help text and does not print the last letter "e" and the last line break.

function Test_UTF_8() ÄÖÜäöüß This text is not printed on
consol>>

Georg Wiora <gwiora>
Sat 08 Dec 2018 12:16:26 PM UTC, comment #10: 

The "äöü" example works for me with the SYSTEM encoding when I test with a recent version from the default branch. I am still un-sure whether it will work for everyone cause I have some local modifications to relevant packages in MXE Octave.

However, "help sym" displays a lot of scrambled characters. That is because that file is encoded in UTF-8 but we assume it to be encoded in the SYSTEM encoding. Converting it from SYSTEM (CP1252 in my case) to UTF-8 creates these odd characters.

That conversion is done in input.cc in function "file_reader::get_input".
Can we differentiate between .m files from the core or packages (which probably always are UTF-8) on the one hand and user created .m files (which could have any encoding) on the other hand at that point? Does the lexer keep track of this?

Markus Mützel <mmuetzel>
Project Member
Fri 07 Dec 2018 03:03:08 PM UTC, comment #9: 

This works correctly for me in 4.4.1 if I use UTF-8 as the .m file encoding.

However, if I set the .m file encoding to SYSTEM (the default on Windows, "CP1252" on my German Windows 10), I can reproduce this bug.

There have been some changes how Octave handles the encoding in .m files in the development branch of Octave which will eventually be released as Octave 5.0.
I will try to test whether this particular use case has been fixed already.

In the meantime, you could manually select to use UTF-8 as the encoding of your .m files. But if you want to use your .m files both in Octave and in Matlab, keep in mind that Matlab uses the SYSTEM codepage on Windows.

Markus Mützel <mmuetzel>
Project Member
Fri 07 Dec 2018 11:05:41 AM UTC, comment #8: 

I can not reproduce the workaround with "more on" and "more off". But I see that the out stops only if there are UTF8 characters in the documentation section of a function.

While the following command

  printf("äöü\n");

does not stop octave from writing to console, the same string in a function documentation block does:

  % äöü

Using "help" command on a function with such a string on console, stops console output for all further commands.

Georg Wiora <gwiora>
Wed 25 Apr 2018 12:02:15 PM UTC, comment #7: 

With the current version 4.2.2 if still can produce this error. Recently I found a method to "heal" the behavior by entering:

more on
more off

After these commands the console output is available again.

Georg Wiora <gwiora>
Mon 15 Jan 2018 09:33:11 AM UTC, comment #6: 

Thank you for your testing hints! I tried the following

1. Can you try making a new m-file by copying say "sym.m" to "mwe.m" and repeating your test?

  • The bug still occurs at the same position, octave of course complains "warning: function name 'sym' does not agree with function filename"

2. You can also try removing the "@seealso" line and see what happens.
*This does not change the behaviour.

3. Can we delete most stuff from mwe.m to find the smallest possible example that shows this bug?

  • Removing all UTF-8 characters: No change
  • *The following did change the behaviour:* Changing document encoding declaration in line 21 from

  %% @documentencoding UTF-8

to

  %% @documentencoding ASCII

After this change the error does not occur any more, but help does print a lot of error messages of the following kind:

ascii "\xE2" does not map to Unicode at /usr/share/texinfo/Texinfo/Parser.pm line 1959, <FH> line 114.

4. Does the "no-newline-after-help" happen with e.g., "help qr" as well?

  • No, it does not show up with "help qr".
Georg Wiora <gwiora>
Mon 15 Jan 2018 05:53:24 AM UTC, comment #5: 

I wonder if this shows up with symbolic because it uses lots of unicode chars / UTF-8 in its docs...

1.  Can you try making a new m-file by copying say "sym.m" to "mwe.m" and repeating your test? 

2.  You can also try removing the "@seealso" line and see what happens.

3.  Can we delete most stuff from mwe.m to find the smallest possible example that shows this bug?

4.  Does the "no-newline-after-help" happen with e.g., "help qr" as well?

(In my limited experience on Windows, a lot of unicode stuff is broken---various open issues about this.  I can't remember if "help sym" is one of them.)

Colin Macdonald <cbm>
Thu 07 Dec 2017 10:48:01 AM UTC, comment #4: 

Oops. Sorry for the syntax error. In my test I did not have this. I used a string in single quotes like

printf('test\n');

I entered/copied all commands to the GUI console. I have not tried them in the script.
When I switched on the diary. All commands and all output lines show up in the diary file.

This is what I see in the diary file (I left out the parts shown by [...]:

>>   printf("This text shows up in my console\n");
This text shows up in my console
>>   help sym
'sym' is a function from the file C:\Octave\OCTAVE~1.1\share\octave\packages\symbolic-2.6.0\@sym\sym.m

-- Constructor on @sym: X = sym (Y)
[...]
     The underlying SymPy string representation (“srepr”) can usually be
     passed directly to ‘sym’: *note @sym/char:: for discussion of the
     details.

     See also: syms, assumptions, @sym/assume, @sym/assumeAlso.

Additional help for built-in functions and operators is
available in the online version of the manual.  Use the command
'doc <topic>' to search the manual index.

Help and information about Octave is also available on the WWW
at http://www.octave.org and via the help@octave.org
mailing list.
>>  printf("This text doesn't show up in my console\n");
This text doesn't show up in my console
>> diary off

I found that the console output stops directly after the string "@sym/asssumeAlso" as shown in my previous post. In the diary file output continues with a dot "." and more text lines produced by the help command.

The help function and the source of the sym command help should be the right places to identify the root of the bug.

Georg Wiora <gwiora>
Thu 07 Dec 2017 01:45:31 AM UTC, comment #3: 

There is an error in your third line of code.  This may be related to the problem.  The apostrophe in "doesn't" ends the text string that you begin with a single quote so that Octave is seeing

  printf('This text doesn'
  t show up in my console\n');

which is an error.  Try using double quotes so the sequence of commands is now:

  printf("This text shows up in my console\n");
  help sym
  printf("This text doesn't show up in my console\n");

Second question, how are you executing these commands?  Are you copying and pasting them into the Command Window or are they in an m-file?

If I put the commands into a file cmds.m and run that it works correctly.  I do appear to have a problem when pasting in to the GUI.  In that case, the last command is not recognized.  I can tell that by using "history 10" to look at the last commands and only "help sym" is in the buffer.

Rik <rik5>
Project Administrator
Wed 06 Dec 2017 08:15:21 AM UTC, comment #2: 

Thank you for your recommendations. I tried __run_test_suite__ without having the effect of lost output. But I finally found a way to reproduce it. I have to enter the following commands after starting octave:

  printf('This text shows up in my console\n');
  help sym
  printf('This text doesn't show up in my console\n');

The console output ends with the following lines:

...
     The underlying SymPy string representation (“srepr”) can usually be
     passed directly to ‘sym’: *note @sym/char:: for discussion of the
     details.

     See also: syms, assumptions, @sym/assume, @sym/assumeAlso>>

The first printf does print to console while the second doesn't. I still don't think this bug is specific to symbolic 2.6.0 package, but does occur in other situations too. I frequently ran into it before starting to use symbolic.

Maybe it is usefull to know, that "help sym" does not print a carriage return after last line of help text. The input prompt is placed directly after the last help output. This may be the place where the output breaks.

For completeness I tried the same with geometry-3.0.0 package: Here I could not reproduce the effect.

  printf('This text shows up in my console\n');
  help points2d
  printf('This text shows up in my console\n');

Georg Wiora <gwiora>
Tue 05 Dec 2017 06:57:21 PM UTC, comment #1: 

If I just open Octave and issue an freport I get:

octave:1> freport

  number  mode  arch       name
  ------  ----  ----       ----
     0     r    ieee-le    stdin
     1     w    ieee-le    stdout
     2     w    ieee-le    stderr

The difference between my report and yours are the two fids python-in and python-out.  Presumably these are pipes used for communication with python that were created by the symbolic package.  It may be that python or the symbolic package is messing around with stdout and stderr and where they are connected (they should be connected to the console at all times).

What happens if you don't load the symbolic package, or other external code, and use Octave?  For example, try running the built-in test suite to stress the interpreter with

__run_test_suite__

After running the test suite, can you execute commands correctly and have the output show up in the console window?

It is also possible that the console code for Microsoft Windows isn't very good.

Rik <rik5>
Project Administrator
Thu 30 Nov 2017 08:58:19 AM UTC, original submission:  

I have frequently problems getting console output in the octave-GUI console window. I can not give a minimal example to produce this effect. Currently I see this effect when working with the symbolic package but I got this effect in other not well defined situations too.

I tried to analyze what happens: After console output was lost I switched on the diary and called freport. In my case it showed the following state in the diary file only.

> freport

  number  mode  arch       name
  ------  ----  ----       ----
     0     r    ieee-le    stdin
     1     w    ieee-le    stdout
     2     w    ieee-le    stderr
     3     r    ieee-le    python-in
     4     w    ieee-le    python-out

For me this points in the direction of the connection between octave core's console output and GUI console. I guess this sometimes breaks.

Georg Wiora <gwiora>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

No files currently attached

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by rik5 (Updated the item)
  • -email is unavailable- added by mmuetzel (Posted a comment)
  • -email is unavailable- added by cbm (Posted a comment)
  • -email is unavailable- added by gwiora (Submitted the item)
  •  

    Do you think this task is very important?
    If so, you can add your encouragement to it.
    This task has 0 encouragements so far.

    Only project members can vote.

     

     

     

    Follow 5 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2019-03-03 rik5 Priority5 - Normal => 2
    2019-01-17 mmuetzel SummaryOctave stops writing to console => Octave stops writing to console on Windows 7
    2018-12-10 rik5 Carbon-CopyRemoved 72865 => -
    2018-12-08 mmuetzel Release4.4.1 => dev
    2018-12-07 mmuetzel Release4.2.1 => 4.4.1

    Back to the top


    Powered by Savane 3.5