Gstreamer + Decklink + Nvidia

Hi!
I'm trying to build some really low latency pipeline with gstreamer and decklink for streaming of captured video in real time.
For this purposes I created simple pipeline:

Code: Select all

gst-launch-1.0 decklinkvideosrc device-number=0 connection=1 drop-no-signal-frames=true buffer-size=2 ! \
videoconvert ! \
capsfilter caps="video/x-raw,format=NV12" ! \
nvh264enc ! \
capsfilter caps="video/x-h264,profile=baseline" ! \
h264parse config-interval=1 ! \
mpegtsmux ! \
rtpmp2tpay ! \
udpsink host=239.239.239.3 port=8889 auto-multicast=true sync=false async=false qos=false


And it worked, but the latency were pretty bad (around 300ms)
After some benchmarking of this pipeline, it was found that 85% of all time is consumed by videoconvert. Which leads to the idea of:
1) Trying to move video conversion to gpu also
2) Remove it and force decklink to output the format acceptable by nvh264enc
Unfortunately, I failed at second
So decided to go with first solution.
As a possibility I chose usage of cudaconvert and cudaupload plugin and on paper - working perfectly.
But in reality, one more time failure, this time on runtime, and can't be catched by gstreamer on element linkage.
Resulted pipeline:

Code: Select all

gst-launch-1.0 decklinkvideosrc drop-no-signal-frames=true device-number=0 connection=1 ! \
   capsfilter caps="video/x-raw,format=(string)BGRA, framerate=60/1" ! \
   cudaupload ! \
   cudaconvert ! \
   capsfilter caps="video/x-raw(memory:CUDAMemory),format=(string)BGRA" ! \
   nvh264enc ! \
   capsfilter caps="video/x-h264,profile=baseline" ! \
   h264parse config-interval=1 ! \
   mpegtsmux ! \
   rtpmp2tpay ! \
   udpsink host=239.239.239.3 port=8889 auto-multicast=true sync=false async=false qos=false


Code: Select all

ERROR: from element /GstPipeline:pipeline0/GstDecklinkVideoSrc:decklinkvideosrc0: Internal data stream error.
Additional debug info:
../subprojects/gstreamer/libs/gst/base/gstbasesrc.c(3177): gst_base_src_loop (): /GstPipeline:pipeline0/GstDecklinkVideoSrc:decklinkvideosrc0:
streaming stopped, reason not-negotiated (-4)


This error told me that on runtime, decklink can't give correct output for the element, so not-negotiated error appears.

I'm using decklink mini recorder hd and running all of this stuff on debian 12

Maybe anyone here know how to deal with this kind of problems? Thank you for your time!
Will be happy to share any more information if needed!

I want to update my post here.

Fortunately, I was able to connect it all together, all I needed were:
glupload and glcolorconvert modules instead of cuda ones. Decklink input in my case always UYVY, while nvidia solutions can't process this color format.

Also the latency problem were not in videoconvert module as I thought at first.

Maybe anyone can share minimum latency they were able to get with decklink and gstreamer video streaming in local network and what pipeline do you used?