bugGnash - The GNU Flash player - Bugs: bug #19832, Animation and Sound out of Sync

 
 

bug #19832: Animation and Sound out of Sync

Submitted by:  Martin Guy <martinwguy>
Submitted on:  Tue May 8 23:13:22 2007  
 
Category: soundSeverity: 3 - Normal
Release: NoneStatus: Fixed
Privacy: PublicAssigned to: Benjamin Wolsey <bwy>
Open/Closed: Closed

Add a New Comment (Rich MarkupRich Markup):
   

You are not logged in

Please log in, so followups can be emailed to you.

 

(Jump to the original submission Jump to the original submission)

Thu Jul 28 12:39:46 2011, comment #16:

This is definitely fixed with Benjamin's work on streaming sound sync. As of 2746c4b51f7b7d7912b29cc178c17393d6928f08.

Sandro Santilli <strk>
Project Member
Mon May 25 17:25:22 2009, comment #15:

Ok, altought Bono is enjoyable, interactive games sucks with this
as too many frames are dropped.

I'll commit the clean patch with compile-time disabled frame dropping and take more time to think about other cleanups.

In particular, another thing to take into account is that
we don't always need to frame-drop, but only when a streaming
sound is present, which makes it horrible to loose sync.

Sandro Santilli <strk>
Project Member
Mon May 25 17:16:47 2009, comment #14:

Alright, next version is cleaner. Moves the frame drop code
to hosting app (Gui) so also helps debugging, and eventually
can be made a run-time option.

(file #18201)

Sandro Santilli <strk>
Project Member
Mon May 25 16:34:14 2009, comment #13:

The attached patch, which applies against revno 10916
implements frames drop in movie_root.
With this I can enjoy AV sync in this movie:
http://www.funny-base.com/cartoons/u2_has.swf

Tests welcome with other movies.

The worst case should be NOTHING is rendered as gnash
is always late.

Feedback welcome.

(file #18200)

Sandro Santilli <strk>
Project Member
Mon May 25 15:56:13 2009, comment #12:

As of revno 10916 AV sync is not a problem for Bozzetto movies anymore. Also the stillalive.php case works fine.

The u2 case still shows bad problems in that gnash is unable
to keep up with the required FPS from the rendering thread,
even trying to catch-up when rendering is not too hard.

I don't think we want to slow down audio though, so the
best we can do is improve rendering speed and eventually
skip rendering when lateness is above a threshold.
I might try that next actually...

Sandro Santilli <strk>
Project Member
Tue Nov 18 21:30:01 2008, comment #11:

A couple more examples:
http://www.vgcats.com/comics/extras/stillalive.php
http://www.newgrounds.com/portal/view/428928

I suspect all flash animation desyncs from video, it's just more obvious with some than others.

Anonymous
 Spam posted by an anonymous
Thu Oct 18 11:03:03 2007, comment #10:

Just tried with GST, same wrong behaviour, so this is not SDL-specific.

Sandro Santilli <strk>
Project Member
Thu Oct 18 10:40:58 2007, comment #9:

A testcase which is easy to tell when it goes out of sync:

http://www.funny-base.com/cartoons/u2_has.swf

SWF6
md5: 1b60ff0055bd5cc4c4c021ce97bacbda

Sandro Santilli <strk>
Project Member
Thu Oct 4 09:11:18 2007, comment #8:

Just an update. I added rendering-skip-when-late support in gnash CVS. It is disabled by default and can be enabled with a compile-time define in gui/gui.h (SKIP_RENDERING_IF_LATE).

It uses a wall-clock timer, but doesn't seem to fix out-of-sync
problems in things like the ninja movie (see bug #20866).
I belive the FPS is not used the proper way (use fps-debugging against a simple movie to verify).

Testing welcome.

Sandro Santilli <strk>
Project Member
Wed Aug 22 05:22:05 2007, comment #7:

Yes, the other player drops video frames to stay synced with the audio, so we probably should too.

Rob Savoye <rsavoye>
Project Administrator
Tue Aug 21 17:50:10 2007, comment #6:

Note that ffmpeg play of elvis.swf isn't bad, just the gst one.
Is GST sound being threaded ?

Sandro Santilli <strk>
Project Member
Tue Aug 21 17:48:35 2007, comment #5:

To recap again. The 0.8.1 branch is normally out of sync, while head has an hack that tries to keep sound back to get in sync.

Playing elvis.swf head sucks :)

I think we should really skip rendering frames if we're too late
on the timeline and keep sound go. I also think this is what
the proprietary player does.

Sandro Santilli <strk>
Project Member
Thu Jun 21 04:21:59 2007, comment #4:

What appears to happen is that it assumes the system timer and sound card timer run in perfect sync. The sound card doesn't usually measure a full second at the same time the system clock does (it's usually off by a few milliseconds or less), and over time this difference becomes greater and more noticeable. And it also doesn't seem to take into account loading times.

You can see it in animations that load big images in the middle of the animation. The audio keeps playing, but the video freezes for a few moments, then continues as if nothing had happened. The audio and video will remain entirely out of sync.

And in the same file, replaying it so the loaded images remain loaded (removing the load stutter) keeps the audio and video playing at proper speed, but they'll slowly go more and more out of sync over time.

A good test case for this can be seen here:
http://clairvoyance.game-server.cc/nightmare.html
The first play through has stuttering video when images load which cause major desyncing. On replay, the video doesn't stutter, but the audio and video slowly drift apart (there's also some rendering anomalies).

Anonymous
Thu May 24 21:14:46 2007, comment #3:

No, this have not been fixed. Just to recap: The problem is that the audio is played in a separate thread, and therefore not sync with video. We just assume that both audio and animation follow the timeline correctly - which is a bit optimistic.

Tomas Groth <tgc>
Thu May 24 20:31:35 2007, comment #2:

Thomas, did you commit some improvements for this ? (I seem to remember something like this)

Sandro Santilli <strk>
Project Member
Wed May 9 01:17:12 2007, comment #1:

Test file attach failed.
http://freaknet.org/martin/video/Bozzetto -> Yes&No

Martin Guy <martinwguy>
Tue May 8 23:13:22 2007, original submission:

agg-sdl opengl-sdl anc cairo-sdl on Bozzetto's Yes&No run the video ahead of the sound. The sound runs correct speed though.

This was with ffmpeg, but I think the frame rate governing code is in backend/sdl.cpp using realtime clock and and SDL usleep-alike. In an ideal world it would sync with the audio timing.

Martin Guy <martinwguy>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach File(s):
   
   
Comment:
   

Attached Files
file #18201:  DropRenderingFrames2.patch added by strk (6kB - text/x-diff)
file #18200:  DropRenderingFrames.patch added by strk (636B - text/x-diff)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -unavailable- added by rsavoye (Posted a comment)
  • -unavailable- added by tgc (Posted a comment)
  • -unavailable- added by strk (Updated the item)
  • -unavailable- added by martinwguy (Submitted the item)
  •  

    Do you think this task is very important?
    If so, you can click here to add your encouragement to it.
    This task has 0 encouragements so far.

    Only logged-in users can vote.

     

    Please enter the title of George Orwell's famous dystopian book (it's a date):

     

     

    Follow 16 latest changes.

    Date Changed By Updated Field Previous Value => Replaced By
    Thu Jul 28 12:39:46 2011strkStatusIn Progress=>Fixed
      Assigned tostrk=>bwy
      Open/ClosedOpen=>Closed
    Mon May 25 17:16:47 2009strkAttached File-=>Added DropRenderingFrames2.patch, #18201
    Mon May 25 16:34:14 2009strkAttached File-=>Added DropRenderingFrames.patch, #18200
      StatusConfirmed=>In Progress
      Assigned toNone=>strk
    Mon May 25 15:56:13 2009strkAssigned totgc=>None
    Thu Oct 18 11:03:03 2007strkCategorysound-sdl=>sound
    Fri Aug 17 18:02:13 2007strkCategorygui-sdl=>sound-sdl
      Severity2 - Minor=>3 - Normal
      StatusNone=>Confirmed
      Assigned toNone=>tgc
    Tue May 29 21:42:14 2007strkSummaryAnimation runs faster than it should=>Animation and Sound out of Sync
    Thu May 24 20:31:35 2007strkCarbon-Copy-=>Added tgc
    Sat May 12 07:09:42 2007strkSeverity3 - Normal=>2 - Minor

    Back to the top


    Powered by Savane 3.1-cleanup