Extend the internal CUDA frames context (CUDAFramesContext) to manage
block-linear CUarray surfaces alongside the existing pitched-linear
allocations.
When the frames context format is AV_PIX_FMT_CUARRAY, the caller
(e.g. NVDEC) can fill cuarray_desc and cuarray_num_surfaces before
calling av_hwframe_ctx_init(), which then allocates the CUarray
surfaces.
Also add cuda_transfer_data_from_cuarray() to support hwdownload
from CUarray frames to host memory via cuMemcpy2DAsync.
Co-authored-by: Diego de Souza <ddesouza@nvidia.com>
Signed-off-by: Diego de Souza <ddesouza@nvidia.com>
Add a new hardware pixel format representing opaque block-linear CUDA
arrays (CUarray). This format is used by NVDEC when decoding into
opaque surfaces registered with cuvidRegisterDecodeSurfaces, and can
be consumed directly by NVENC as CUDA array input without requiring
a pitch-linear copy.
Signed-off-by: Diego de Souza <ddesouza@nvidia.com>
Convert packed 24/32-bit RGB/BGR/RGBA/BGRA input to PAL8 using a per-frame palette
whose colors are placed on a face-centered cubic lattice (realized as the
scaled D3/D4 checkerboard lattice), with a user-supplied density controlling
the number of lattice steps spanning one color axis.
Only lattice points actually used by a frame enter its palette; if a frame
needs more than 256 of them, the filter will itearatively drop palettte
entries and reassign affected pixels until 256 color remain
lookup uses the Conway-Sloane rounding algorithm. Supported dithering
modes: none, ordered 8x8 bayer (swscale), Cluster & Void blue noise and
Floyd-Steinberg error diffusion.
Co-Authored-by: Fable-5
Signed-off-by: Michael Niedermayer <michael@niedermayer.cc>
Useful for formats where they provide the downmix matrix in the bistream rather
than scale factors for channel groups.
Signed-off-by: James Almer <jamrial@gmail.com>
vf_libplacebo is unique in that it can function as both a Vulkan filter, and a
software filter, depending on how it the filter graph is configured. While this
is very flexible, it create a problem in situations where the filter does not
receive a Vulkan hw frames context up front. When that doesn't happen, it will
initialise its own standalone pl context, and then fail to interoperate with
a context provided via the input link. This then leads to graph failures.
There are two primary scenarios where this existing logic breaks what should
be valid configurations:
* When the global filter hw device is a cuda device and we use something like
`hwupload=derive_device=vulkan` to pass frames from cuda to vulkan
* In mpv (and probably other media players) which don't set the global filter
hw device at all. In this case, it's impossible to configure vf_libplacebo
to use a hw frames ctx, even if it's using Vulkan for everythng else. This
prevents the use of vf_libplacebo in any fully hardware accelerated pipeline
in mpv
There are various ways we could imagine addressing it - such as allowing the
filter to discard the initial pl context and recreating it based on the passed
in device, but it's easier to reason about if we add a flag that explicitly
tells the filter that it should inherit the device context from the input link.
This puts the filter into a mode that works like all the other Vulkan filters.
This requires explicit configuration from the user, but the intent is clearer,
and the user can always know when it's necessary as they define the filter
graph.
This commit attempts to address feedback from this filter's users, by
introducing a new option which controls the amelioration mechanism.
The "blend" option is a factor which is multiplied by the difference
in badness (between the threshold and the currently accumulated
badness). This difference normally controls how much of the next frame
we can let through without making it exceed the badness threshold.
Setting the option to zero effectively puts the filter into a mode
where it always duplicates the last frame (which did not put
accumulated badness over the threshold) instead of attempting to blend
in new frames. I have received reports that this mode is preferable
to users for some types of media.
Add a new boolean option -update_filemtime to the image2 muxer that
sets each output file's modification time based on the creation_time
metadata plus the frame's PTS offset.
This is useful when extracting frames from dashcam or action camera
footage where wall-clock timestamps should be preserved on the output
files, allowing photo management tools to sort frames by capture time
without post-processing.
The option requires creation_time metadata to be set (via -metadata
creation_time=...). If not present, a warning is logged and the
option is silently disabled. When PTS is unavailable, the creation
time is used as-is without frame offset.
Uses utimes() on POSIX and _utime() on Windows to set file timestamps
with microsecond and second precision respectively.
Includes a FATE roundtrip test that writes frames with a known
creation_time, reads them back using the demuxer's -ts_from_file
option, and verifies the PTS values match the expected timestamps.
Closes: https://code.ffmpeg.org/FFmpeg/FFmpeg/issues/22537
Signed-off-by: marcos ashton <marcosashiglesias@gmail.com>
Similar to -retry_errors, this retries blocks whose only crime is failing
the CRC self-check.
Sponsored-by: nxtedition AB
Signed-off-by: Niklas Haas <git@haasn.dev>
Add a muxer that wraps encoded image in the iTerm2 inline image protocol
(OSC 1337) so ffmpeg can play video directly in an iTerm2 terminal. The
output is a self-contained byte stream: it can be played live or saved
to a file and replayed with cat.
By turning it into a tristate that defaults to -1 (auto). Users can still
force a particular value for debugging.
Sponsored-by: nxtedition AB
Signed-off-by: Niklas Haas <git@haasn.dev>
write_header_trailer, individual_header_trailer, segment_header_filename and
segment_wrap_number had no documentation. Describe each option and the ways
they constrain one another.
Signed-off-by: Bogdan Lisman <bogdan@pydevsolutions.com>
This may be faster or slower than the existing specialized kernels,
so I opted not to prefer it by default. I also deliberately didn't expose
additional filter function capabilites yet.
The main motivating reason here is to get correct anti-aliasing behavior
when downscaling, which is currently completely broken.
Signed-off-by: Niklas Haas <git@haasn.dev>
Deprecated in commit 09c53a04c5
on 2022-06-11.
Thanks to Michael Niedermayer for pointing out that
the documentation needs to be updated, too.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
The decision to switch to checking peer certificates by default
at the next major version bump was announced on 2025-08-09
in commit 5621eee672.
Thanks to Michael Niedermayer for pointing out that the documentation
needs to be updated, too.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
libnpp and the corresponding filters have been deprecated
in commit 994a368451
on 2025-09-26. By the time of our next release,
a year will have passed, so they are removed immediately.
Note: Passing --enable-libnpp to configure results in
a warning about the deprecation and is otherwise a no-op.
Signed-off-by: Andreas Rheinhardt <andreas.rheinhardt@outlook.com>
The shaping option is now exposed by both the ass and subtitles filters.
Document it under subtitles, update the ass filter description to refer
to the shared option set, and note that complex shaping is required for
Arabic, Hebrew, Devanagari and Thai and depends on a HarfBuzz-enabled
libass build.
Signed-off-by: Jun Zhao <barryjzhao@tencent.com>
This patch adds ONNX Runtime as a new DNN backend for FFmpeg's dnn_processing
filter, enabling hardware-accelerated neural network inference on multiple
GPU and NPU platforms.
Execution Providers Supported:
- CPU execution provider (default)
- CUDA execution provider (NVIDIA GPUs)
- DirectML execution provider (AMD/Intel/NVIDIA GPUs on Windows)
- VitisAI execution provider (AMD Ryzen AI NPU)
The options for dnn_processing with dnn_backend=onnx:
- device: execution provider — cpu, cuda, dml, or vitisai (default: cpu)
- device_id: GPU device index (default: 0)
- threads_per_operation: inference thread count for CPU EP (default: 0, auto)
- input: input tensor name. When omitted the backend resolves it from loaded session
- output: output tensor name. When omitted the backend resolves it from loaded session
Example usage:
# CPU inference
ffmpeg -i input.mp4 -vf "format=rgb24,dnn_processing=dnn_backend=onnx:model=model.onnx:input=image_in:output=image_out" output.mp4
# CUDA GPU inference
ffmpeg -i input.mp4 -vf "dnn_processing=dnn_backend=onnx:model=model.onnx:device=cuda:device_id=0" output.mp4
# DirectML GPU inference (Windows)
ffmpeg -i input.mp4 -vf "dnn_processing=dnn_backend=onnx:model=model.onnx:device=dml:device_id=0" output.mp4
# VitisAI NPU inference
ffmpeg -i input.mp4 -vf "dnn_processing=dnn_backend=onnx:model=model.onnx:device=vitisai" output.mp4
Note: depending on the model, you may need a format filter (e.g. format=rgb24 or format=grayf32) before dnn_processing to convert the frames to the pixel format the model's input tensor expects.
Signed-off-by: younengxiao <steven.xiao@amd.com>
Reviewed-by: Guo Yejun <yejun.guo@intel.com>
The lrc muxer has a precision option controlling the number of
fractional digits written in each timestamp, but it was not documented.
Add it to the lrc section, including its range and default.
Signed-off-by: Bogdan Lisman <bogdan@pydevsolutions.com>
This value is matched to the typical seek latency in a reasonably capable
7200 rpm disk device, as well as the typical latency of an on-premise HTTP
request.
Note that this change should rarely have a significant effect, because
it only matters when using multiple concurrent processes, and one process
is somehow stuck in I/O (or died). Since we sleep in a loop for 1/16th of
the requested timeout value, this should only increase the effective read
latency by up to ~500 us on top of the actual underlying latency.
The alternative is hammering the same underlying resource with the exact
same requests at the exact same time (e.g. during init).
Sponsored-by: nxtedition AB
Signed-off-by: Niklas Haas <git@haasn.dev>
This will effectively disable the cache but allows the cache layer to verify
cached files against the original input file. Useful only for debugging
the shared cache protocol itself, as file corruption can already be caught by
the CRC check.
This adds a new protocol shared:URI which is distinct from the existing
`cache:` in that it is explicity designed to be thread-safe and cross-process,
enabling multiple ffmpeg processes (or multiple ffmpeg decoders within the same
process) to share a single cache file, for e.g. a remote HTTP stream. As such,
it uses a radically different internal design.
To facilitate zero-knowledge cross-process interoperability, the cache file
itself is just a memory-mapped representation of the underlying file data,
which has the side benefit that the resulting cache file will contain a
working copy of the streamed file (assuming the stream was read to
completion).
To keep track of which regions are cached and which are not, we use a
secondary file that contains a minimal header along with a static bytemap of
blocks within the file. This secondary file is also used to store metadata
such as the filesize, if known, as well as marking "failed" blocks.
Both files can grow dynamically in order to accommodate larger/growing files,
and can be atomically updated (through the use of shared space maps). I have
extensively checked the space map initalization and update code for race
conditions, and I believe the current design to be solid.
That said, it is the user's responsibility to some extent to ensure that the
same URI is not used for different streams, as we rely on the URI to uniquely
identify the cache files. That said, we use a cryptographic hash with
sufficient collision resistance to protect against possible abuse. The lack of
any implicit default on `-cache_dir` also means that `shared:` can't be enabled
via URL injection to possibly access random files on the disk (or intentionally
leak content from other streams with similar URIs, even if the cryptograhic
hash function is broken).
If the input is expected to grow, we shouldn't make any assumptions about
the file size. This matches e.g. the behavior of streamed protocols like
chunked HTTP, which similarly return ENOSYS for streams of unknown size.
Sponsored-by: nxtedition AB
Signed-off-by: Niklas Haas <git@haasn.dev>
This allows constraining the set of available backends. This serves as a
better replacement for the "unstable" flag, which is a bit ambiguous. Allows
users to, for example, opt into the memcpy or x86 backend, while excluding
e.g. the upcoming JIT backends.
Signed-off-by: Niklas Haas <git@haasn.dev>
These are needed for interop with e.g. libplacebo, which needs to know the
correct flags to call vkGetDeviceQueue2.
Signed-off-by: Niklas Haas <git@haasn.dev>
StachowiakDawid
Romain Beauxis
Timo RothenpielerandDiego de Souza
Michael NiedermayerandFable-5
Araz Iusubov
Lynne
James Almer
Vignesh Venkat
Philip Langdale
Vladimir Panteleev
marcos ashton
Niklas Haas
Andreas Rheinhardt
Zhao Zhili
Bogdan Lisman
Jun Zhao
stevxiao
Daniel Verkamp
Bejoy
Kacper Michajłow