Our great sponsors
-
-
easyeffects
Limiter, compressor, convolver, equalizer and auto volume and many other plugins for PipeWire applications
-
InfluxDB
Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
-
I regularly hit a couple.
This one breaks audio and video in virtual machines:
https://gitlab.freedesktop.org/pipewire/pipewire/-/issues/46...
The effects of this one vary:
https://gitlab.freedesktop.org/pipewire/wireplumber/-/merge_...
For me it breaks audio in Chrome and VLC:
https://github.com/wwmm/easyeffects/issues/1789
Usually playback will be locked to a hardware clock, typically the audio clock. This does mean that the audio clock and frame clock will drift over time, as they're not locked to each other; it doesn't matter how accurate the clocks are, they will drift. This is one source of dropped/doubled frames. It's possible to sync to the video framerate instead, of course, though audio desync artifacts tend to be more noticeable and objectionable than video ones. You can handle this by dynamically measuring the clock drift and variably resampling the audio, but this is relatively complicated and computationally expensive. Generally this type of drift isn't too noticeable, these clocks are usually well within 100ppm, which would result in 1 frame slip every few minutes.
The issue with framerate mismatch is a fundamentally different one, in that it's not a synchronization problem. It's the result of a non-integral ratio between display frames and playback frames, and exists regardless of clock source. So it's the harder of the two to deal with, and is worse in magnitude too at several slipped frames per minute with e.g. 29.976fps vs. 30fps. In the classical approach, this simply results in frames getting doubled or skipped, since it's not tracking the video rate at all, it just asks for the display to be updated at its frame-time, and the display system will either get a chance to show it or not. The slightly more refined approach is to play back at the closest integral ratio (e.g. play 29.976fps video at 30fps on a 60fps display), and resample the audio to match this new rate, but again this is computationally expensive and complicated.
mpv team has a good writeup: https://github.com/mpv-player/mpv/wiki/Display-synchronizati...