vf_libplacebo is unique in that it can function as both a Vulkan filter, and a
software filter, depending on how it the filter graph is configured. While this
is very flexible, it create a problem in situations where the filter does not
receive a Vulkan hw frames context up front. When that doesn't happen, it will
initialise its own standalone pl context, and then fail to interoperate with
a context provided via the input link. This then leads to graph failures.
There are two primary scenarios where this existing logic breaks what should
be valid configurations:
* When the global filter hw device is a cuda device and we use something like
`hwupload=derive_device=vulkan` to pass frames from cuda to vulkan
* In mpv (and probably other media players) which don't set the global filter
hw device at all. In this case, it's impossible to configure vf_libplacebo
to use a hw frames ctx, even if it's using Vulkan for everythng else. This
prevents the use of vf_libplacebo in any fully hardware accelerated pipeline
in mpv
There are various ways we could imagine addressing it - such as allowing the
filter to discard the initial pl context and recreating it based on the passed
in device, but it's easier to reason about if we add a flag that explicitly
tells the filter that it should inherit the device context from the input link.
This puts the filter into a mode that works like all the other Vulkan filters.
This requires explicit configuration from the user, but the intent is clearer,
and the user can always know when it's necessary as they define the filter
graph.
libplacebo versions before v365 passed .flags = 0 when retrieving the queues
from imported Vulkan devices, so we have to error out in the case of a mismatch
to avoid undefined behavior (Vulkan spec).
See-Also: https://code.videolan.org/videolan/libplacebo/-/merge_requests/856
Sponsored-by: nxtedition AB
Signed-off-by: Niklas Haas <git@haasn.dev>
ff_scale_adjust_dimensions() can now return a negative error code when
the evaluated output dimensions are non-positive. Check the return
value and fail fast instead of continuing with the unadjusted result.
Signed-off-by: Jun Zhao <barryjzhao@tencent.com>
Otherwise, this will indefinitely persist the last couple of mapped frames
(including any extra decoded frames) in memory, even though they will never be
used again, causing a gradual memory leak until filter uninit.
Signed-off-by: Niklas Haas <git@haasn.dev>
Sponsored-by: nxtedition AB
Add chroma_location option so that, in the subsampled-to-subsampled case, the destination's chroma siting can be changed from the source's without having to use other filters. Useful, for example, when converting BT.2020 to BT.709, where the former customarily uses "top left" and the latter "left."
Update documentation.
Closes: #21185
MSVC build was failing due to preprocessor directives being used within function-like macro argument lists, which is undefined behavior according to the C11 spec. Fixed by replacing the directive blocks with macro definitions outside of the macro invocations.
In commit 6e0034ab7e, the image crop adjustment
was moved after the fitting logic. However, this moved the adjustment inside the
`if (src == ref)` branch, thus missing applying the same un-rotation to input
frames that are *not* the reference frame.
Fix this by pulling the logic back out of the branch again. While we could just
move it after the fitting logic, I think it's more clear to the intent of the
code to just preserve the (rotated) crop rect as a separate variable
`crop_orig`.
Fixes: 6e0034ab7e
When combining rotation with a FIT_ mode other than FIT_FILL, the fitting
logic was operating on the un-rotated rects, when it should have been
operating on the rotated (output) rects.
This is a regression introduced by the addition of the rotation option,
which overrode the existing rotation attribute that may have been set to
the image.
To fix it, add the rotation istead of setting it - however we have to do this
directly when mapping, so as to not add it multiple times.
Fixes: 4f623b4c59
This allows choosing whether the `fit_mode` merely controls the placement
of the image within the output resolution, or whether the output resolution
is also adjusted according to the given `fit_mode`.
The semantics of these keywords are well-defined by the CSS 'object-fit'
property. This is arguably more user-friendly and less obtuse than the
existing `normalize_sar` and `pad_crop_ratio` options. Additionally, this
comes with two new (useful) behaviors, `none` and `scale_down`, neither of
which map elegantly to the existing options.
One additional benefit of this option is that, unlike `normalize_sar`, it
does *not* also imply `reset_sar`; meaning that users can now choose to
have an anamorphic base layer and still have the overlay images scaled to fit
on top of it according to the chosen strategy.
See-Also: https://drafts.csswg.org/css-images/#the-object-fit
Blending onto independent alpha framebuffers is not possible under the
constraints of the supported blend operators. While we could handle
blending premul-onto-premul, this would break if the base layer is YUV,
since premultiplied alpha does not survive the (nonlinear) YUV conversion.
Fortunately, blending independent-onto-premul is just as easy, and works in
all cases. So just force this mode when using a linear intermediate blend
texture, which is always RGBA.
Instead of directly mutating `opts->params`. Avoids any possible leak of
overriden params between invocations of this function, as well as the later
`pl_render_image` during the linear output pass.
Chooses the desired output alpha mode. Note that this depends on
an upstream version of libplacebo new enough to respect the corresponding
AVFrame field in pl_map_avframe_ex.
In constant FPS mode, it's possible for there no be *no* input active at
the current timestamp, even though there would be active inputs later in
the timeline (nb_active > 0). This would previously hit the AVERROR_BUG case.
With this change, we can instead output an empty frame.
Instead of having a single s->status field to track the most recently
EOF'd stream, track the number of active streams directly and only query
the most recent status at the end.
The reason this worked before is because we implicitly relied on the
assumption that `ok` would always be true until all streams are EOF, but
because it's possible to have gaps in the timeline when mixing multiple
streams, this is not always the case in principle.
In practice, this fixes a bug where the filter would early-exit (EOF)
too soon, when any input reached EOF and there is a gap in the timeline.
When using libplacebo to composite multiple streams with complex PTS
values, it's possible for there to be a "hole" in the output stream; in
particular in constant FPS mode. This commit refactors output_frame() to
allow it to handle the case of there being no active input.
pl_frame_mix_current() will return NULL if all frames are in the future,
but when libplacebo is using a frame mixer with a radius greater than 1,
future frames are expected to already be partially renderered. Instead, use
pl_frame_mix_nearest(), which is guaranteed to give us a valid frame for any
nonempty frame mix.
Fixes: 3091bca3ed
Instead of undconditionally using the first input. This covers the case of
one layer fully obscuring another layer, in which case that should become
the new "base" layer.
This prevents leaking stale metadata from previous frames, for example if
an overlay temporarily obscures this input and then un-obscures it again. It
is worth pointing out that this does change the semantics subtly, because of
the smoothing period on detected HDR metadata, but I argue that the new
behavior is an improvement, as it will avoid leaking past metadata that is
definitely no longer relevant after an image is unobscured.
Sometimes, one input fully obscures another. In this case, we can skip
actually rendering any input below the obscuring one.
The reason I don't simply start the main render loop at `idx_start` will
become apparent in the following commit.
We can't use pl_frame_is_cropped() on this dummy frame, but we need to
determine the reference frame before we can map the real output, so to
resolve this conflict, we just reimplement the crop detection logic using
the output link dimensions.
It is possible for pl_queue_update() to return PL_QUEUE_OK, but to generate
an empty frame mix. This happens if the first frame of that input is in the
future.
In this case, we should skip an input as not active, similar to inputs that
have already reached EOF.
Instead of copying over the entire target and changing a few fields,
set the entire struct to a whitelist of safe properties that we want to
persist on the intermediate texture.
In particular, this avoids leaking irrelevant state related to the
acquire/release callbacks, e.g., which can otherwise cause deadlocks
when the same vulkan frame is attempted to be acquired twice.
This gives vastly improved blending results than when blending directly in
the desired output colorspace. Overridable by the existing "disable_linear"
option.
This is functionally similar to combining multiple "libplacebo" filters,
but does not rely on the existence of a Vulkan filter link, so it can be used
without performance penalty in all circumstances. It's also enabled by
default, without requiring special action from the user.
This was requested by users of `vf_libplacebo`, to mirror the existing
option on the other `vf_scale_*` family of filters. While we have
`vf_normalize`, it was not as useful in the event that the content
stretching was actually desired.
Bridges an important usability gap between `vf_scale` and `vf_libplacebo`
that made mixing and matching the filters needlessly difficult.
This aligns the behavior of vf_libplacebo with other filters in the
vf_scale family, that forward any change in the SAR as a result of
changing the output resolution to the output frame / link, and updates
the ff_scale_adjust_dimensions() call to continue working as intended.
The new behavior reflects the documentation of vf_libplacebo, which
described this behavior despite that not being the way the filter worked
up to this point.
As an aside, also fixes a bug where the AVFrame SAR was inconsistent
with the AVFilterLink SAR when the s->nb_inputs > 1 condition was met.
Under normal circumstances, this change does not affect anything, as the vast
majority of filters either support only vulkan or only software formats.
However, when a filter supports both (such as vf_libplacebo itself, and
possibly vf_scale in the future), linking together two such filter instances
without an explicit format will default matching the input format, resulting
in a redundant round trip through host RAM.
This change bumps up AV_PIX_FMT_VULKAN to the first entry in the format list,
ensuring that it gets preferred whenever possible.
Lynne
Philip Langdale
Kacper Michajłow
Robert Nagy
Niklas Haas
Jun Zhao
Zhao Zhili
GeoffreyAA
nikitalita
Timo Rothenpieler