bugGnash - The GNU Flash player - Bugs: bug #19832, Animation and Sound out of Sync

 
 

bug #19832: Animation and Sound out of Sync

Submitter:  Martin Guy <martinwguy>
Submitted:  Tue 08 May 2007 11:13:22 PM UTC
   
 
Category:  sound Severity:  3 - Normal
Release:  None Status:  Fixed
Privacy:  Public Assigned to:  bwy
Open/Closed:  Closed
* Mandatory Fields

Add a New Comment Rich Markup
   

Jump to the original submission

Thu 28 Jul 2011 12:39:46 PM UTC, comment #17: 

This is definitely fixed with Benjamin's work on streaming sound sync. As of 2746c4b51f7b7d7912b29cc178c17393d6928f08.

Sandro Santilli <strk>
Group Member
Mon 25 May 2009 05:25:22 PM UTC, comment #16: 

Ok, altought Bono is enjoyable, interactive games sucks with this
as too many frames are dropped.

I'll commit the clean patch with compile-time disabled frame dropping and take more time to think about other cleanups.

In particular, another thing to take into account is that
we don't always need to frame-drop, but only when a streaming
sound is present, which makes it horrible to loose sync.


Sandro Santilli <strk>
Group Member
Mon 25 May 2009 05:16:47 PM UTC, comment #15: 

Alright, next version is cleaner. Moves the frame drop code
to hosting app (Gui) so also helps debugging, and eventually
can be made a run-time option.

(file #18201)

Sandro Santilli <strk>
Group Member
Mon 25 May 2009 04:34:14 PM UTC, comment #14: 

The attached patch, which applies against revno 10916
implements frames drop in movie_root.
With this I can enjoy AV sync in this movie:
http://www.funny-base.com/cartoons/u2_has.swf

Tests welcome with other movies.

The worst case should be NOTHING is rendered as gnash
is always late.

Feedback welcome.

(file #18200)

Sandro Santilli <strk>
Group Member
Mon 25 May 2009 03:56:13 PM UTC, comment #13: 

As of revno 10916 AV sync is not a problem for Bozzetto movies anymore. Also the stillalive.php case works fine.

The u2 case still shows bad problems in that gnash is unable
to keep up with the required FPS from the rendering thread,
even trying to catch-up when rendering is not too hard.

I don't think we want to slow down audio though, so the
best we can do is improve rendering speed and eventually
skip rendering when lateness is above a threshold.
I might try that next actually...

Sandro Santilli <strk>
Group Member
Tue 18 Nov 2008 09:30:01 PM UTC, comment #12: 

A couple more examples:
http://www.vgcats.com/comics/extras/stillalive.php
http://www.newgrounds.com/portal/view/428928

I suspect all flash animation desyncs from video, it's just more obvious with some than others.

Anonymous
  Spam posted by anonymous
Thu 18 Oct 2007 11:03:03 AM UTC, comment #10: 

Just tried with GST, same wrong behaviour, so this is not SDL-specific.

Sandro Santilli <strk>
Group Member
Thu 18 Oct 2007 10:40:58 AM UTC, comment #9: 

A testcase which is easy to tell when it goes out of sync:

http://www.funny-base.com/cartoons/u2_has.swf

SWF6
md5: 1b60ff0055bd5cc4c4c021ce97bacbda

Sandro Santilli <strk>
Group Member
Thu 04 Oct 2007 09:11:18 AM UTC, comment #8: 

Just an update. I added rendering-skip-when-late support in gnash CVS. It is disabled by default and can be enabled with a compile-time define in gui/gui.h (SKIP_RENDERING_IF_LATE).

It uses a wall-clock timer, but doesn't seem to fix out-of-sync
problems in things like the ninja movie (see bug #20866).
I belive the FPS is not used the proper way (use fps-debugging against a simple movie to verify).

Testing welcome.

Sandro Santilli <strk>
Group Member
Wed 22 Aug 2007 05:22:05 AM UTC, comment #7: 

Yes, the other player drops video frames to stay synced with the audio, so we probably should too.

Rob Savoye <rsavoye>
Group administrator
Tue 21 Aug 2007 05:50:10 PM UTC, comment #6: 

Note that ffmpeg play of elvis.swf isn't bad, just the gst one.
Is GST sound being threaded ?

Sandro Santilli <strk>
Group Member
Tue 21 Aug 2007 05:48:35 PM UTC, comment #5: 

To recap again. The 0.8.1 branch is normally out of sync, while head has an hack that tries to keep sound back to get in sync.

Playing elvis.swf head sucks :)

I think we should really skip rendering frames if we're too late
on the timeline and keep sound go. I also think this is what
the proprietary player does.

Sandro Santilli <strk>
Group Member
Thu 21 Jun 2007 04:21:59 AM UTC, comment #4: 

What appears to happen is that it assumes the system timer and sound card timer run in perfect sync. The sound card doesn't usually measure a full second at the same time the system clock does (it's usually off by a few milliseconds or less), and over time this difference becomes greater and more noticeable. And it also doesn't seem to take into account loading times.

You can see it in animations that load big images in the middle of the animation. The audio keeps playing, but the video freezes for a few moments, then continues as if nothing had happened. The audio and video will remain entirely out of sync.

And in the same file, replaying it so the loaded images remain loaded (removing the load stutter) keeps the audio and video playing at proper speed, but they'll slowly go more and more out of sync over time.

A good test case for this can be seen here:
http://clairvoyance.game-server.cc/nightmare.html
The first play through has stuttering video when images load which cause major desyncing. On replay, the video doesn't stutter, but the audio and video slowly drift apart (there's also some rendering anomalies).

Anonymous
Thu 24 May 2007 09:14:46 PM UTC, comment #3: 

No, this have not been fixed. Just to recap: The problem is that the audio is played in a separate thread, and therefore not sync with video. We just assume that both audio and animation follow the timeline correctly - which is a bit optimistic.

Tomas Groth <tgc>
Thu 24 May 2007 08:31:35 PM UTC, comment #2: 

Thomas, did you commit some improvements for this ? (I seem to remember something like this)

Sandro Santilli <strk>
Group Member
Wed 09 May 2007 01:17:12 AM UTC, comment #1: 

Test file attach failed.
http://freaknet.org/martin/video/Bozzetto -> Yes&No

Martin Guy <martinwguy>
Tue 08 May 2007 11:13:22 PM UTC, original submission:  

agg-sdl opengl-sdl anc cairo-sdl on Bozzetto's Yes&No run the video ahead of the sound. The sound runs correct speed though.

This was with ffmpeg, but I think the frame rate governing code is in backend/*sdl*.cpp using realtime clock and and SDL usleep-alike. In an ideal world it would sync with the audio timing.

Martin Guy <martinwguy>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach Files:
   
   
Comment:
   

Attached Files
file #18201:  DropRenderingFrames2.patch added by strk (6KiB - text/x-diff)
file #18200:  DropRenderingFrames.patch added by strk (636B - text/x-diff)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -email is unavailable- added by rsavoye (Posted a comment)
  • -email is unavailable- added by tgc (Posted a comment)
  • -email is unavailable- added by strk (Updated the item)
  • -email is unavailable- added by martinwguy (Submitted the item)
  •  

    There are 0 votes so far. Votes easily highlight which items people would like to see resolved in priority, independently of the priority of the item set by tracker managers.

    Only logged-in users can vote.

     

    Follow 16 latest changes.

    Date Changed by Updated Field Previous Value => Replaced by
    2011-07-28 strk StatusIn Progress Fixed
        Assigned tostrk bwy
        Open/ClosedOpen Closed
    2009-05-25 strk Attached File- Added DropRenderingFrames2.patch, #18201
    2009-05-25 strk Attached File- Added DropRenderingFrames.patch, #18200
        StatusConfirmed In Progress
        Assigned toNone strk
    2009-05-25 strk Assigned totgc None
    2007-10-18 strk Categorysound-sdl sound
    2007-08-17 strk Categorygui-sdl sound-sdl
        Severity2 - Minor 3 - Normal
        StatusNone Confirmed
        Assigned toNone tgc
    2007-05-29 strk SummaryAnimation runs faster than it should Animation and Sound out of Sync
    2007-05-24 strk Carbon-Copy- Added tgc
    2007-05-12 strk Severity3 - Normal 2 - Minor

    Back to the top

    Powered by Savane 3.13-f8d8.
    Corresponding source code