bugGnash - The GNU Flash player - Bugs: bug #47054, `getvariable.as` test produces...

 
 

bug #47054: `getvariable.as` test produces different result on different Flash Player version

Submitter:  Nutchanon Wetchasit <nachanon>
Submitted:  Wed 03 Feb 2016 12:59:17 PM UTC
   
 
Category:  testsuite Severity:  3 - Normal
Priority:  5 - Normal Release:  master
Status:  None Privacy:  Public
Assigned to:  None Open/Closed:  Open
* Mandatory Fields

Post a Comment

Add a New Comment Rich Markup
   

Discussion

Thu 17 Mar 2016 05:48:31 AM UTC, comment #4: 

When tried on native version of Flash Player each SWF is supposed to run:

Text output of `getvariable-v6.swf` under Flash Player 6.0 r79 NPAPI:

 - xtrace enabled -
SWF6 - WIN 6,0,79,0

[getvariable.as master-22343-e705394]
#passed: 58
#failed: 0
#total tests run: 58


Text output of `getvariable-v7.swf` under Flash Player 7.0 r73 NPAPI:

 - xtrace enabled -
SWF7 - WIN 7,0,73,0

[getvariable.as master-22343-e705394]
#passed: 58
#failed: 0
#total tests run: 58


Text output of `getvariable-v8.swf` under Flash Player 8.0 r42 NPAPI:

 - xtrace enabled -
SWF8 - WIN 8,0,42,0

[getvariable.as master-22343-e705394]
#passed: 58
#failed: 0
#total tests run: 58


Although Flash 5 is not tested yet (there're some problems on how
DejaGNU functions used TextField), according to the results so far,
this issue is actually Adobe Flash Player's own regression,
which is introduced somewhere between version 9.x and version 11.x.

In order to pinpoint the exact version (for documentation),
a regression test would need to be performed.

Gnash testsuite: 0.8.11dev (git e705394 29-Jan-2016)
Browser: Mozilla Firefox 11.0
System: Microsoft Windows XP Professional SP3

Nutchanon Wetchasit <nachanon>
Wed 03 Feb 2016 02:33:37 PM UTC, comment #3: 

There is an issue in current Gnash testsuite that prevent SWF5 test files
from running under a real Flash Player 5.x; see bug #47055.

Nutchanon Wetchasit <nachanon>
Wed 03 Feb 2016 01:06:01 PM UTC, comment #2: 

But when I tried them on older Flash Player version, interesting things happened...

Text output of `getvariable-v5.swf` under Flash Player 9.0.45.0:

 - xtrace enabled -
SWF5 - WIN 9,0,45,0

[getvariable.as master-22343-e705394]
#passed: 52
#failed: 0
#total tests run: 52


Text output of `getvariable-v6.swf` under Flash Player 9.0.45.0:

 - xtrace enabled -
SWF6 - WIN 9,0,45,0

[getvariable.as master-22343-e705394]
#passed: 58
#failed: 0
#total tests run: 58


Text output of `getvariable-v7.swf` under Flash Player 9.0.45.0:

 - xtrace enabled -
SWF7 - WIN 9,0,45,0

[getvariable.as master-22343-e705394]
#passed: 58
#failed: 0
#total tests run: 58


Text output of `getvariable-v8.swf` under Flash Player 9.0.45.0:

 - xtrace enabled -
SWF8 - WIN 9,0,45,0

[getvariable.as master-22343-e705394]
#passed: 58
#failed: 0
#total tests run: 58


You would see that the tests are now passing. So, this is likely to be
Adobe Flash Player's fault (not testsuite's bug). But to confirm,
one would need to test each SWF on Flash Player version it's native to.

Flash Player: 9.0 r45 (9.0.45.0) Standalone
System: Microsoft Windows XP Professional SP3

Nutchanon Wetchasit <nachanon>
Wed 03 Feb 2016 01:01:57 PM UTC, comment #1: 

When I run them on Microsoft Windows, results are the same...

Text output of `getvariable-v5.swf` under Flash Player 11.3.300.271:

 - xtrace enabled -
SWF5 - WIN 11,3,300,271

[getvariable.as master-22343-e705394]
FAILED: expected: 5.4 obtained:  [./getvariable.as:105]
#passed: 51
#failed: 1
#total tests run: 52


Text output of `getvariable-v6.swf` under Flash Player 11.3.300.271:

 - xtrace enabled -
SWF6 - WIN 11,3,300,271

[getvariable.as master-22343-e705394]
FAILED: expected: 5.4 obtained:  [./getvariable.as:105]
#passed: 57
#failed: 1
#total tests run: 58


Text output of `getvariable-v7.swf` under Flash Player 11.3.300.271:

 - xtrace enabled -
SWF7 - WIN 11,3,300,271

[getvariable.as master-22343-e705394]
FAILED: expected: 5.4 obtained: undefined [./getvariable.as:105]
#passed: 57
#failed: 1
#total tests run: 58


Text output of `getvariable-v8.swf` under Flash Player 11.3.300.271:

 - xtrace enabled -
SWF8 - WIN 11,3,300,271

[getvariable.as master-22343-e705394]
FAILED: expected: 5.4 obtained: undefined [./getvariable.as:105]
#passed: 57
#failed: 1
#total tests run: 58


Gnash testsuite: 0.8.11dev (git e705394 29-Jan-2016)
Flash Player: 11.3 r300 (11.3.300.271) NPAPI binary
Browser: Opera 11.51
System: Microsoft Windows XP Professional SP3

Nutchanon Wetchasit <nachanon>
Wed 03 Feb 2016 12:59:17 PM UTC, original submission:  

This is a follow-up to @strk's comment on failing test in bug #45840.
According to several experiment I have made, the issue might not be actually
a bug in testsuite; more like Flash Player's own regression.

But I think Gnash testsuite's code documention could be improved to
document this regression; providing warning for user who tries to run
the test under recent Adobe Flash Player version in the future.

After I run getvariable.as test from Gnash testusite under Adobe Flash Player 11.
I found that one check in the test is failing:

Text output of `getvariable-v5.swf` under Flash Player 11.2.202.491:

 - xtrace enabled -
SWF5 - LNX 11,2,202,491

[getvariable.as master-22343-e705394]
FAILED: expected: 5.4 obtained:  [./getvariable.as:105]
#passed: 51
#failed: 1
#total tests run: 52


Text output of `getvariable-v6.swf` under Flash Player 11.2.202.491:

 - xtrace enabled -
SWF6 - LNX 11,2,202,491

[getvariable.as master-22343-e705394]
FAILED: expected: 5.4 obtained:  [./getvariable.as:105]
#passed: 57
#failed: 1
#total tests run: 58


Text output of `getvariable-v7.swf` under Flash Player 11.2.202.491:

 - xtrace enabled -
SWF7 - LNX 11,2,202,491

[getvariable.as master-22343-e705394]
FAILED: expected: 5.4 obtained: undefined [./getvariable.as:105]
#passed: 57
#failed: 1
#total tests run: 58


Text output of `getvariable-v8.swf` under Flash Player 11.2.202.491:

 - xtrace enabled -
SWF8 - LNX 11,2,202,491

[getvariable.as master-22343-e705394]
FAILED: expected: 5.4 obtained: undefined [./getvariable.as:105]
#passed: 57
#failed: 1
#total tests run: 58


`getvariable-v5.swf`, `getvariable-v6.swf`, `getvariable-v7.swf`,
`getvariable-v8.swf`, and `Dejagnu.swf` are built from current Gnash
git version codebase.

Gnash testsuite: 0.8.11dev (git e705394 29-Jan-2016)
Flash Player: 11.2 r202 (11.2.202.491) Standalone
System: Debian GNU/Linux 7.0 Wheezy i386

Nutchanon Wetchasit <nachanon>

 

Attached Files

This item currently has no attached files.

(Note: upload size limit is set to 4.0MiB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

 

Dependencies

This item does not depend on any other items.

No items depend on this one.

 

Mail Notification Carbon-Copy List

Carbon-Copy List
  • -email is unavailable- added by nachanon (Submitted the item)
  •  

    Votes

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only logged-in users can vote.

     

    History

    No changes have been made to this item

    Back to the top

    Powered by Savane 3.16-ed84.
    Corresponding source code