bugGnash - The GNU Flash player - Bugs: bug #19832, Animation and Sound out of Sync

 
 

bug #19832: Animation and Sound out of Sync

Submitted by:  Martin Guy <martinwguy>
Submitted on:  Tue 08 May 2007 11:13:22 PM UTC  
 
Category: soundSeverity: 3 - Normal
Release: NoneStatus: Fixed
Privacy: PublicAssigned to: Benjamin Wolsey <bwy>
Open/Closed: Closed

Add a New Comment (Rich MarkupRich Markup):
   

You are not logged in

Please log in, so followups can be emailed to you.

 

(Jump to the original submission Jump to the original submission)

Thu 28 Jul 2011 12:39:46 PM UTC, comment #16:

This is definitely fixed with Benjamin's work on streaming sound sync. As of 2746c4b51f7b7d7912b29cc178c17393d6928f08.

Sandro Santilli <strk>
Project Member
Mon 25 May 2009 05:25:22 PM UTC, comment #15:

Ok, altought Bono is enjoyable, interactive games sucks with this
as too many frames are dropped.

I'll commit the clean patch with compile-time disabled frame dropping and take more time to think about other cleanups.

In particular, another thing to take into account is that
we don't always need to frame-drop, but only when a streaming
sound is present, which makes it horrible to loose sync.

Sandro Santilli <strk>
Project Member
Mon 25 May 2009 05:16:47 PM UTC, comment #14:

Alright, next version is cleaner. Moves the frame drop code
to hosting app (Gui) so also helps debugging, and eventually
can be made a run-time option.

(file #18201)

Sandro Santilli <strk>
Project Member
Mon 25 May 2009 04:34:14 PM UTC, comment #13:

The attached patch, which applies against revno 10916
implements frames drop in movie_root.
With this I can enjoy AV sync in this movie:
http://www.funny-base.com/cartoons/u2_has.swf

Tests welcome with other movies.

The worst case should be NOTHING is rendered as gnash
is always late.

Feedback welcome.

(file #18200)

Sandro Santilli <strk>
Project Member
Mon 25 May 2009 03:56:13 PM UTC, comment #12:

As of revno 10916 AV sync is not a problem for Bozzetto movies anymore. Also the stillalive.php case works fine.

The u2 case still shows bad problems in that gnash is unable
to keep up with the required FPS from the rendering thread,
even trying to catch-up when rendering is not too hard.

I don't think we want to slow down audio though, so the
best we can do is improve rendering speed and eventually
skip rendering when lateness is above a threshold.
I might try that next actually...

Sandro Santilli <strk>
Project Member
Tue 18 Nov 2008 09:30:01 PM UTC, comment #11:

A couple more examples:
http://www.vgcats.com/comics/extras/stillalive.php
http://www.newgrounds.com/portal/view/428928

I suspect all flash animation desyncs from video, it's just more obvious with some than others.

Anonymous
 Spam posted by an anonymous
Thu 18 Oct 2007 11:03:03 AM UTC, comment #10:

Just tried with GST, same wrong behaviour, so this is not SDL-specific.

Sandro Santilli <strk>
Project Member
Thu 18 Oct 2007 10:40:58 AM UTC, comment #9:

A testcase which is easy to tell when it goes out of sync:

http://www.funny-base.com/cartoons/u2_has.swf

SWF6
md5: 1b60ff0055bd5cc4c4c021ce97bacbda

Sandro Santilli <strk>
Project Member
Thu 04 Oct 2007 09:11:18 AM UTC, comment #8:

Just an update. I added rendering-skip-when-late support in gnash CVS. It is disabled by default and can be enabled with a compile-time define in gui/gui.h (SKIP_RENDERING_IF_LATE).

It uses a wall-clock timer, but doesn't seem to fix out-of-sync
problems in things like the ninja movie (see bug #20866).
I belive the FPS is not used the proper way (use fps-debugging against a simple movie to verify).

Testing welcome.

Sandro Santilli <strk>
Project Member
Wed 22 Aug 2007 05:22:05 AM UTC, comment #7:

Yes, the other player drops video frames to stay synced with the audio, so we probably should too.

Rob Savoye <rsavoye>
Project Administrator
Tue 21 Aug 2007 05:50:10 PM UTC, comment #6:

Note that ffmpeg play of elvis.swf isn't bad, just the gst one.
Is GST sound being threaded ?

Sandro Santilli <strk>
Project Member
Tue 21 Aug 2007 05:48:35 PM UTC, comment #5:

To recap again. The 0.8.1 branch is normally out of sync, while head has an hack that tries to keep sound back to get in sync.

Playing elvis.swf head sucks :)

I think we should really skip rendering frames if we're too late
on the timeline and keep sound go. I also think this is what
the proprietary player does.

Sandro Santilli <strk>
Project Member
Thu 21 Jun 2007 04:21:59 AM UTC, comment #4:

What appears to happen is that it assumes the system timer and sound card timer run in perfect sync. The sound card doesn't usually measure a full second at the same time the system clock does (it's usually off by a few milliseconds or less), and over time this difference becomes greater and more noticeable. And it also doesn't seem to take into account loading times.

You can see it in animations that load big images in the middle of the animation. The audio keeps playing, but the video freezes for a few moments, then continues as if nothing had happened. The audio and video will remain entirely out of sync.

And in the same file, replaying it so the loaded images remain loaded (removing the load stutter) keeps the audio and video playing at proper speed, but they'll slowly go more and more out of sync over time.

A good test case for this can be seen here:
http://clairvoyance.game-server.cc/nightmare.html
The first play through has stuttering video when images load which cause major desyncing. On replay, the video doesn't stutter, but the audio and video slowly drift apart (there's also some rendering anomalies).

Anonymous
Thu 24 May 2007 09:14:46 PM UTC, comment #3:

No, this have not been fixed. Just to recap: The problem is that the audio is played in a separate thread, and therefore not sync with video. We just assume that both audio and animation follow the timeline correctly - which is a bit optimistic.

Tomas Groth <tgc>
Thu 24 May 2007 08:31:35 PM UTC, comment #2:

Thomas, did you commit some improvements for this ? (I seem to remember something like this)

Sandro Santilli <strk>
Project Member
Wed 09 May 2007 01:17:12 AM UTC, comment #1:

Test file attach failed.
http://freaknet.org/martin/video/Bozzetto -> Yes&No

Martin Guy <martinwguy>
Tue 08 May 2007 11:13:22 PM UTC, original submission:

agg-sdl opengl-sdl anc cairo-sdl on Bozzetto's Yes&No run the video ahead of the sound. The sound runs correct speed though.

This was with ffmpeg, but I think the frame rate governing code is in backend/sdl.cpp using realtime clock and and SDL usleep-alike. In an ideal world it would sync with the audio timing.

Martin Guy <martinwguy>

 

(Note: upload size limit is set to 16384 kB, after insertion of the required escape characters.)

Attach File(s):
   
   
Comment:
   

Attached Files
file #18201:  DropRenderingFrames2.patch added by strk (6kB - text/x-diff)
file #18200:  DropRenderingFrames.patch added by strk (636B - text/x-diff)

 

Depends on the following items: None found

Items that depend on this one: None found

 

Carbon-Copy List
  • -unavailable- added by rsavoye (Posted a comment)
  • -unavailable- added by tgc (Posted a comment)
  • -unavailable- added by strk (Updated the item)
  • -unavailable- added by martinwguy (Submitted the item)
  •  

    Do you think this task is very important?
    If so, you can click here to add your encouragement to it.
    This task has 0 encouragements so far.

    Only logged-in users can vote.

     

    Please enter the title of George Orwell's famous dystopian book (it's a date):

     

     

    Follow 16 latest changes.

    Date Changed By Updated Field Previous Value => Replaced By
    Thu 28 Jul 2011 12:39:46 PM UTCstrkStatusIn Progress=>Fixed
      Assigned tostrk=>bwy
      Open/ClosedOpen=>Closed
    Mon 25 May 2009 05:16:47 PM UTCstrkAttached File-=>Added DropRenderingFrames2.patch, #18201
    Mon 25 May 2009 04:34:14 PM UTCstrkAttached File-=>Added DropRenderingFrames.patch, #18200
      StatusConfirmed=>In Progress
      Assigned toNone=>strk
    Mon 25 May 2009 03:56:13 PM UTCstrkAssigned totgc=>None
    Thu 18 Oct 2007 11:03:03 AM UTCstrkCategorysound-sdl=>sound
    Fri 17 Aug 2007 06:02:13 PM UTCstrkCategorygui-sdl=>sound-sdl
      Severity2 - Minor=>3 - Normal
      StatusNone=>Confirmed
      Assigned toNone=>tgc
    Tue 29 May 2007 09:42:14 PM UTCstrkSummaryAnimation runs faster than it should=>Animation and Sound out of Sync
    Thu 24 May 2007 08:31:35 PM UTCstrkCarbon-Copy-=>Added tgc
    Sat 12 May 2007 07:09:42 AM UTCstrkSeverity3 - Normal=>2 - Minor

    Back to the top


    Powered by Savane 3.1-cleanup