Messages in this thread | | | Date | Wed, 6 Apr 2022 14:40:55 +0200 | Subject | Re: [PATCH v4 00/15] Move HEVC stateless controls out of staging | From | Benjamin Gaignard <> |
| |
Le 06/04/2022 à 14:28, Adam Ford a écrit : > On Wed, Apr 6, 2022 at 1:56 AM Benjamin Gaignard > <benjamin.gaignard@collabora.com> wrote: >> >> Le 05/04/2022 à 23:27, Adam Ford a écrit : >>> On Mon, Apr 4, 2022 at 10:56 AM Benjamin Gaignard >>> <benjamin.gaignard@collabora.com> wrote: >>>> Le 02/04/2022 à 18:59, Adam Ford a écrit : >>>>> On Sat, Apr 2, 2022 at 11:22 AM Adam Ford <aford173@gmail.com> wrote: >>>>>> On Fri, Apr 1, 2022 at 8:18 AM Benjamin Gaignard >>>>>> <benjamin.gaignard@collabora.com> wrote: >>>>>>> Le 31/03/2022 à 08:53, Benjamin Gaignard a écrit : >>>>>>>> Le 30/03/2022 à 20:52, Adam Ford a écrit : >>>>>>>>> On Wed, Mar 30, 2022 at 2:53 AM Benjamin Gaignard >>>>>>>>> <benjamin.gaignard@collabora.com> wrote: >>>>>>>>>> Le 28/02/2022 à 15:08, Benjamin Gaignard a écrit : >>>>>>>>>>> This series aims to make HEVC uapi stable and usable for hardware >>>>>>>>>>> decoder. HEVC uapi is used by 2 mainlined drivers (Cedrus and Hantro) >>>>>>>>>>> and 2 out of the tree drivers (rkvdec and RPI). >>>>>>>>>>> >>>>>>>>>>> After the remarks done on version 2, I have completely reworked to >>>>>>>>>>> patches >>>>>>>>>>> split so changelogs are meaningless. I have also drop "RFC" from the >>>>>>>>>>> titles. >>>>>>>>>>> >>>>>>>>>>> Version 4: >>>>>>>>>>> - Add num_entry_point_offsets field in struct >>>>>>>>>>> v4l2_ctrl_hevc_slice_params >>>>>>>>>>> - Fix V4L2_CID_STATELESS_HEVC_ENTRY_POINT_OFFSETS name >>>>>>>>>>> - Initialize control V4L2_CID_STATELESS_HEVC_ENTRY_POINT_OFFSETS >>>>>>>>>>> - Fix space/tab issue in kernel-doc >>>>>>>>>>> - Add patch to change data_bit_offset definition >>>>>>>>>>> - Fix hantro-media SPDX license >>>>>>>>>>> - put controls under stateless section in v4l2-ctrls-defs.c >>>>>>>>>>> >>>>>>>>>>> At the end fluster tests results on IMX8MQ is 77/147 for HEVC codec. >>>>>>>>>> Dear reviewers, >>>>>>>>>> >>>>>>>>>> This series is waiting for your feedback, >>>>>>>>> I tried several times with the suggested repos for both the kernel and >>>>>>>>> g-streamer without success getting Fluster to pass any tests on the >>>>>>>>> imx8mq. I can try again but I likely won't get to it until this >>>>>>>>> weekend. If I can get it working, I'll test both the 8mq and 8mm. >>>>>>>> Thanks a lot for that. >>>>>>>> >>>>>>>> Benjamin >>>>>>> Adam, >>>>>>> >>>>>>> You may need to check if h265parse and v4l2slh265dec are available on your board. >>>>>> I ran gst-inspect to see what showed up with 265 in the name. >>>>>> >>>>>> # gst-inspect-1.0 |grep 265 >>>>>> libav: avdec_h265: libav HEVC (High Efficiency Video Coding) decoder >>>>>> rtp: rtph265depay: RTP H265 depayloader >>>>>> rtp: rtph265pay: RTP H265 payloader >>>>>> typefindfunctions: video/x-h265: h265, x265, 265 >>>>>> v4l2codecs: v4l2slh265dec: V4L2 Stateless H.265 Video Decoder >>>>>> videoparsersbad: h265parse: H.265 parser >>>>>> >>>>>> It appears I have both h265parse and v4l2slh265dec. >>>>>> >>>>>>> fluster check if v4l2slh265dec is working fine with this command line: >>>>>>> >>>>>>> gst-launch-1.0 appsrc num-buffers=0 ! h265parse ! v4l2slh265dec ! fakesink >>>>>>> >>>>>>> so if one of them is missing it won't work. >>>>>> gst-launch-1.0 appsrc num-buffers=0 ! h265parse ! v4l2slh265dec ! fakesink >>>>>> Setting pipeline to PAUSED ... >>>>>> 0:00:00.098389938 526 0xaaaaf9d86ac0 ERROR v4l2codecs-decoder >>>>>> gstv4l2decoder.c:725:gst_v4l2_decoder_get_controls:<v4l2decoder2> >>>>>> VIDIOC_G_EXT_CTRLS failed: Invalid argument >>>>>> ERROR: from element >>>>>> /GstPipeline:pipeline0/v4l2slh265dec:v4l2slh265dec0: Driver did not >>>>>> report framing and start code method. >>>>>> Additional debug info: >>>>>> ../subprojects/gst-plugins-bad/sys/v4l2codecs/gstv4l2codech265dec.c(155): >>>>>> gst_v4l2_codec_h265_dec_open (): >>>>>> /GstPipeline:pipeline0/v4l2slh265dec:v4l2slh265dec0: >>>>>> gst_v4l2_decoder_get_controls() failed: Invalid argument >>>>>> ERROR: pipeline doesn't want to preroll. >>>>>> ERROR: from element >>>>>> /GstPipeline:pipeline0/v4l2slh265dec:v4l2slh265dec0: Could not >>>>>> initialize supporting library. >>>>>> Additional debug info: >>>>>> ../subprojects/gst-plugins-base/gst-libs/gst/video/gstvideodecoder.c(2909): >>>>>> gst_video_decoder_change_state (): >>>>>> /GstPipeline:pipeline0/v4l2slh265dec:v4l2slh265dec0: >>>>>> Failed to open decoder >>>>>> ERROR: pipeline doesn't want to preroll. >>>>>> Failed to set pipeline to PAUSED. >>>>>> Setting pipeline to NULL ... >>>>>> Freeing pipeline ... >>>>>> >>>>>> Does this mean I have a wrong version of the kernel and/or incomplete patches? >>>>> I double checked the branches. >>>>> >>>>> Kernel: >>>>> https://gitlab.collabora.com/benjamin.gaignard/for-upstream.git >>>>> branch: origin/HEVC_UAPI_V4 >>>>> >>>>> Gstreamer: >>>>> https://gitlab.freedesktop.org/benjamin.gaignard1/gstreamer.git >>>>> branch: origin/benjamin.gaignard1/gstreamer-HEVC_aligned_with_kernel_5.15 >>>>> >>>>> >>>>> I am still not able to run h.265/HEVC tests. >>>> Hello Adam, >>>> >>>> I have updated the following branches with the versions I have used today: >>>> >>>> Kernel: >>>> https://gitlab.collabora.com/benjamin.gaignard/for-upstream.git >>>> branch: origin/HEVC_UAPI_V5 only one change in documentation vs version 4 but rebased in v5.18-rc1 >>>> >>>> Gstreamer: >>>> https://gitlab.freedesktop.org/benjamin.gaignard1/gstreamer.git >>>> branch: origin/benjamin.gaignard1/gstreamer-HEVC_aligned_with_kernel_5.15 updated on the latest GST main branch >>>> >>>> I hope this will work fine this time. >>> I wish I had better news for you: >>> >>> dmesg shows the hantro driver is being loaded: >>> >>> [ 38.612243] hantro-vpu 38300000.video-codec: registered >>> nxp,imx8mq-vpu-g1-dec as /dev/video0 >>> [ 38.612618] hantro-vpu 38310000.video-codec: registered >>> nxp,imx8mq-vpu-g2-dec as /dev/video1 >>> >>> # gst-inspect-1.0 |grep 265 >>> libav: avdec_h265: libav HEVC (High Efficiency Video Coding) decoder >>> rtp: rtph265depay: RTP H265 depayloader >>> rtp: rtph265pay: RTP H265 payloader >>> typefindfunctions: video/x-h265: h265, x265, 265 >>> v4l2codecs: v4l2slh265dec: V4L2 Stateless H.265 Video Decoder >>> videoparsersbad: h265parse: H.265 parser >>> >>> Fluster reports: >>> GStreamer-H.265-V4L2SL-Gst1.0: GStreamer H.265 V4L2SL decoder for >>> GStreamer 1.0... ❌ >> Still the same error about non supported control ? > # gst-lagst-launch-1.0 appsrc num-buffers=0 ! h265parse ! > v4l2slh265dec ! fakesink > Setting pipeline to PAUSED ... > 0:00:01.704385508 420 0xaaaaec0a76c0 ERROR v4l2codecs-decoder > gstv4l2decoder.c:725:gst_v4l2_decoder_get_controls:<v4l2decoder2> > VIDIOC_G_EXT_CTRLS failed: Invalid argument > ERROR: from element > /GstPipeline:pipeline0/v4l2slh265dec:v4l2slh265dec0: Driver did not > report framing and start code method. > Additional debug info: > ../subprojects/gst-plugins-bad/sys/v4l2codecs/gstv4l2codech265dec.c(155): > gst_v4l2_codec_h265_dec_open (): > /GstPipeline:pipeline0/v4l2slh265dec:v4l2slh265dec0: > gst_v4l2_decoder_get_controls() failed: Invalid argument > ERROR: pipeline doesn't want to preroll. > ERROR: from element > /GstPipeline:pipeline0/v4l2slh265dec:v4l2slh265dec0: Could not > initialize supporting library. > Additional debug info: > ../subprojects/gst-plugins-base/gst-libs/gst/video/gstvideodecoder.c(2909): > gst_video_decoder_change_state (): > /GstPipeline:pipeline0/v4l2slh265dec:v4l2slh265dec0: > Failed to open decoder > ERROR: pipeline doesn't want to preroll. > Failed to set pipeline to PAUSED. > Setting pipeline to NULL ... > Freeing pipeline ...
What is the result of v4l2-ctl ? Here I have: v4l2-ctl --all -d 1 Driver Info: Driver name : hantro-vpu Card type : nxp,imx8mq-vpu-g2-dec Bus info : platform: hantro-vpu Driver version : 5.18.0 Capabilities : 0x84204000 Video Memory-to-Memory Multiplanar Streaming Extended Pix Format Device Capabilities Device Caps : 0x04204000 Video Memory-to-Memory Multiplanar Streaming Extended Pix Format Media Driver Info: Driver name : hantro-vpu Model : hantro-vpu Serial : Bus info : platform: hantro-vpu Media version : 5.18.0 Hardware revision: 0x00000000 (0) Driver version : 5.18.0 Interface Info: ID : 0x0300000c Type : V4L Video Entity Info: ID : 0x00000001 (1) Name : nxp,imx8mq-vpu-g2-dec-source Function : V4L2 I/O Pad 0x01000002 : 0: Source Link 0x02000008: to remote pad 0x1000004 of entity 'nxp,imx8mq-vpu-g2-dec-proc': Data, Enabled, Immutable Priority: 2 Format Video Capture Multiplanar: Width/Height : 48/48 Pixel Format : 'VT12' (Y/CbCr 4:2:0 (4x4 Linear)) Field : None Number of planes : 1 Flags : Colorspace : JPEG Transfer Function : Default YCbCr/HSV Encoding: Default Quantization : Default Plane 0 : Bytes per Line : 48 Size Image : 3600 Format Video Output Multiplanar: Width/Height : 48/48 Pixel Format : 'S265' (HEVC Parsed Slice Data) Field : None Number of planes : 1 Flags : Colorspace : JPEG Transfer Function : Default YCbCr/HSV Encoding: Default Quantization : Default Plane 0 : Bytes per Line : 0 Size Image : 4608
User Controls
hantro_hevc_slice_header_skip_b 0x00981a80 (int) : min=0 max=256 step=1 default=0 value=0
Codec Controls
hevc_profile 0x00990b67 (menu) : min=0 max=2 default=0 value=0 0: Main 1: Main Still Picture 2: Main 10 hevc_level 0x00990b68 (menu) : min=0 max=8 default=0 value=0 0: 1 1: 2 2: 2.1 3: 3 4: 3.1 5: 4 6: 4.1 7: 5 8: 5.1
Stateless Codec Controls
vp9_frame_decode_parameters 0x00a40a2c (unknown): type=261 flags=has-payload vp9_probabilities_updates 0x00a40a2d (unknown): type=260 flags=has-payload hevc_sequence_parameter_set 0x00a40a90 (unknown): type=270 flags=has-payload hevc_picture_parameter_set 0x00a40a91 (unknown): type=271 flags=has-payload hevc_scaling_matrix 0x00a40a93 (unknown): type=273 flags=has-payload hevc_decode_parameters 0x00a40a94 (unknown): type=274 flags=has-payload hevc_decode_mode 0x00a40a95 (menu) : min=1 max=1 default=1 value=1 1: Frame-Based hevc_start_code 0x00a40a96 (menu) : min=1 max=1 default=1 value=1 1: Annex B Start Code
> >> Benjamin >> >>> adam >>>> Benjamin >>>> >>>>> adam >>>>>> adam >>>>>>> Regards, >>>>>>> Benjamin >>>>>>> >>>>>>>>> adam >>>>>>>>>> Thanks, >>>>>>>>>> Benjamin >>>>>>>>>> >>>>>>>>>>> Benjamin >>>>>>>>>>> >>>>>>>>>>> >>>>>>>>>>> Benjamin Gaignard (12): >>>>>>>>>>> media: uapi: HEVC: Add missing fields in HEVC controls >>>>>>>>>>> media: uapi: HEVC: Rename HEVC stateless controls with STATELESS >>>>>>>>>>> prefix >>>>>>>>>>> media: uapi: HEVC: Add document uAPI structure >>>>>>>>>>> media: uapi: HEVC: Define V4L2_CID_STATELESS_HEVC_SLICE_PARAMS >>>>>>>>>>> as a >>>>>>>>>>> dynamic array >>>>>>>>>>> media: uapi: Move parsed HEVC pixel format out of staging >>>>>>>>>>> media: uapi: Add V4L2_CID_STATELESS_HEVC_ENTRY_POINT_OFFSETS >>>>>>>>>>> control >>>>>>>>>>> media: uapi: Move the HEVC stateless control type out of staging >>>>>>>>>>> media: controls: Log HEVC stateless control in .std_log >>>>>>>>>>> media: uapi: Create a dedicated header for Hantro control >>>>>>>>>>> media: uapi: HEVC: fix padding in v4l2 control structures >>>>>>>>>>> media: uapi: Change data_bit_offset definition >>>>>>>>>>> media: uapi: move HEVC stateless controls out of staging >>>>>>>>>>> >>>>>>>>>>> Hans Verkuil (3): >>>>>>>>>>> videodev2.h: add V4L2_CTRL_FLAG_DYNAMIC_ARRAY >>>>>>>>>>> v4l2-ctrls: add support for dynamically allocated arrays. >>>>>>>>>>> vivid: add dynamic array test control >>>>>>>>>>> >>>>>>>>>>> .../userspace-api/media/drivers/hantro.rst | 5 - >>>>>>>>>>> .../media/v4l/ext-ctrls-codec-stateless.rst | 833 >>>>>>>>>>> ++++++++++++++++++ >>>>>>>>>>> .../media/v4l/ext-ctrls-codec.rst | 780 >>>>>>>>>>> ---------------- >>>>>>>>>>> .../media/v4l/pixfmt-compressed.rst | 7 +- >>>>>>>>>>> .../media/v4l/vidioc-g-ext-ctrls.rst | 20 + >>>>>>>>>>> .../media/v4l/vidioc-queryctrl.rst | 8 + >>>>>>>>>>> .../media/videodev2.h.rst.exceptions | 5 + >>>>>>>>>>> .../media/test-drivers/vivid/vivid-ctrls.c | 15 + >>>>>>>>>>> drivers/media/v4l2-core/v4l2-ctrls-api.c | 103 ++- >>>>>>>>>>> drivers/media/v4l2-core/v4l2-ctrls-core.c | 198 ++++- >>>>>>>>>>> drivers/media/v4l2-core/v4l2-ctrls-defs.c | 37 +- >>>>>>>>>>> drivers/media/v4l2-core/v4l2-ctrls-priv.h | 3 +- >>>>>>>>>>> drivers/media/v4l2-core/v4l2-ctrls-request.c | 13 +- >>>>>>>>>>> drivers/staging/media/hantro/hantro_drv.c | 27 +- >>>>>>>>>>> drivers/staging/media/hantro/hantro_hevc.c | 8 +- >>>>>>>>>>> drivers/staging/media/sunxi/cedrus/cedrus.c | 24 +- >>>>>>>>>>> .../staging/media/sunxi/cedrus/cedrus_dec.c | 10 +- >>>>>>>>>>> .../staging/media/sunxi/cedrus/cedrus_h265.c | 2 +- >>>>>>>>>>> include/media/hevc-ctrls.h | 250 ------ >>>>>>>>>>> include/media/v4l2-ctrls.h | 48 +- >>>>>>>>>>> include/uapi/linux/hantro-media.h | 19 + >>>>>>>>>>> include/uapi/linux/v4l2-controls.h | 439 +++++++++ >>>>>>>>>>> include/uapi/linux/videodev2.h | 13 + >>>>>>>>>>> 23 files changed, 1697 insertions(+), 1170 deletions(-) >>>>>>>>>>> delete mode 100644 include/media/hevc-ctrls.h >>>>>>>>>>> create mode 100644 include/uapi/linux/hantro-media.h >>>>>>>>>>>
| |