lkml.org 
[lkml]   [2021]   [Nov]   [30]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    /
    From
    Date
    SubjectRe: [RFC 0/5] arm64: imx8mm: Enable Hantro VPUs
    On Tue, Nov 30, 2021 at 1:28 PM Tim Harvey <tharvey@gateworks.com> wrote:
    >
    > On Tue, Nov 30, 2021 at 6:00 AM Ezequiel Garcia
    > <ezequiel@vanguardiasur.com.ar> wrote:
    > >
    > > Hi Tim,
    > >
    > > On Mon, 29 Nov 2021 at 16:36, Tim Harvey <tharvey@gateworks.com> wrote:
    > > >
    > > > On Mon, Nov 29, 2021 at 10:59 AM Adam Ford <aford173@gmail.com> wrote:
    > > ..
    > > > >
    > > >
    > > > Adam,
    > > >
    > > > What deps did you install in order to get v4l2codecs building? I
    > > > installed libgudev-1.0-dev based on Nicolas' suggestion and rebuilt
    > > > (not sure if I needed to re-configure somehow) but there is still
    > > > nothing in build/subprojects/gst-plugins-bad/sys/v4l2codecs/. A 'meson
    > > > configure' tells me that v4l2codecs is set to 'auto' but I'm not sure
    > > > how to find out what dependencies are needed or what may be missing.
    > > >
    > >
    > > At least in my case (Centps-derivative), this is what I've done:
    > >
    > > ...
    > > gst-plugins-bad| Run-time dependency gudev-1.0 found: NO (tried
    > > pkgconfig and cmake)
    > >
    > > Installed gudev ... and then:
    > >
    > > ...
    > > gst-plugins-bad| Dependency gudev-1.0 found: YES 232 (cached)
    > > ...
    > > gst-plugins-bad 1.19.3.1
    > >
    > > Plugins : accurip, adpcmdec, adpcmenc, aiff, asfmux,
    > > audiobuffersplit, audiofxbad, audiomixmatrix, audiolatency,
    > > audiovisualizers, autoconvert, bayer,
    > > camerabin, codecalpha, coloreffects,
    > > debugutilsbad, dvbsubenc, dvbsuboverlay, dvdspu, faceoverlay,
    > > festival, fieldanalysis, freeverb, frei0r,
    > > gaudieffects, gdp, geometrictransform,
    > > id3tag, inter, interlace, ivfparse, ivtc, jp2kdecimator, jpegformat,
    > > rfbsrc, midi, mpegpsdemux,
    > > mpegpsmux, mpegtsdemux, mpegtsmux, mxf,
    > > netsim, rtponvif, pcapparse, pnm, proxy, legacyrawparse,
    > > removesilence, rist, rtmp2, rtpmanagerbad,
    > > sdpelem, segmentclip, siren, smooth,
    > > speed, subenc, switchbin, timecode, transcode, videofiltersbad,
    > > videoframe_audiolevel, videoparsersbad,
    > > videosignal, vmnc, y4mdec, decklink, dvb,
    > > fbdevsink, ipcpipeline, nvcodec, shm, v4l2codecs, hls, sctp
    > >
    > > GStreamer current master build fails. It's a known issue which will be
    > > fixed today:
    > >
    > > [...]
    > > [8/9] Compiling C object
    > > subprojects/gst-plugins-bad/sys/v4l2codecs/libgstv4l2codecs.so.p/gstv4l2codecvp9dec.c.o
    > > FAILED: subprojects/gst-plugins-bad/sys/v4l2codecs/libgstv4l2codecs.so.p/gstv4l2codecvp9dec.c.o
    > > cc -Isubprojects/gst-plugins-bad/sys/v4l2codecs/libgstv4l2codecs.so.p
    > > -Isubprojects/gst-plugins-bad/sys/v4l2codecs
    > > -I../subprojects/gst-plugins-bad/sys/v4l2codecs
    > > -Isubprojects/gst-plugins-bad -I../subprojects/gst-plugins-bad
    > > -Isubprojects/gstreamer/libs -I../subprojects/gstreamer/libs
    > > -Isubprojects/gstreamer -I../subprojects/gstreamer
    > > -Isubprojects/gst-plugins-bad/gst-libs
    > > -I../subprojects/gst-plugins-bad/gst-libs
    > > -Isubprojects/gst-plugins-base/gst-libs
    > > -I../subprojects/gst-plugins-base/gst-libs -Isubprojects/orc
    > > -I../subprojects/orc -Isubprojects/gstreamer/gst
    > > -Isubprojects/gst-plugins-base/gst-libs/gst/video
    > > -Isubprojects/gst-plugins-base/gst-libs/gst/pbutils
    > > -Isubprojects/gst-plugins-base/gst-libs/gst/audio
    > > -Isubprojects/gst-plugins-base/gst-libs/gst/tag
    > > -I/usr/include/glib-2.0 -I/usr/lib64/glib-2.0/include
    > > -I/usr/include/gudev-1.0 -fdiagnostics-color=always
    > > -D_FILE_OFFSET_BITS=64 -Wall -Winvalid-pch -O2 -g -fvisibility=hidden
    > > -fno-strict-aliasing -DG_DISABLE_DEPRECATED -Wmissing-prototypes
    > > -Wdeclaration-after-statement -Wold-style-definition
    > > -Wmissing-declarations -Wredundant-decls -Wwrite-strings -Wformat
    > > -Wformat-security -Winit-self -Wmissing-include-dirs -Waddress
    > > -Wno-multichar -Wvla -Wpointer-arith -fPIC -pthread -DHAVE_CONFIG_H
    > > -MD -MQ subprojects/gst-plugins-bad/sys/v4l2codecs/libgstv4l2codecs.so.p/gstv4l2codecvp9dec.c.o
    > > -MF subprojects/gst-plugins-bad/sys/v4l2codecs/libgstv4l2codecs.so.p/gstv4l2codecvp9dec.c.o.d
    > > -o subprojects/gst-plugins-bad/sys/v4l2codecs/libgstv4l2codecs.so.p/gstv4l2codecvp9dec.c.o
    > > -c ../subprojects/gst-plugins-bad/sys/v4l2codecs/gstv4l2codecvp9dec.c
    > > ../subprojects/gst-plugins-bad/sys/v4l2codecs/gstv4l2codecvp9dec.c:92:3:
    > > error: unknown type name ‘grefcount’
    > > grefcount ref_count;
    > > ^~~~~~~~~
    > > ../subprojects/gst-plugins-bad/sys/v4l2codecs/gstv4l2codecvp9dec.c: In
    > > function ‘gst_v4l2_codec_vp9_dec_picture_data_new’:
    > > ../subprojects/gst-plugins-bad/sys/v4l2codecs/gstv4l2codecvp9dec.c:106:3:
    > > warning: implicit declaration of function ‘g_ref_count_init’; did you
    > > mean ‘g_cond_init’? [-Wimplicit-function-declaration]
    > > g_ref_count_init (&pic_data->ref_count);
    > > ^~~~~~~~~~~~~~~~
    > > g_cond_init
    > > ../subprojects/gst-plugins-bad/sys/v4l2codecs/gstv4l2codecvp9dec.c: In
    > > function ‘gst_v4l2_codec_vp9_dec_picture_data_ref’:
    > > ../subprojects/gst-plugins-bad/sys/v4l2codecs/gstv4l2codecvp9dec.c:118:3:
    > > warning: implicit declaration of function ‘g_ref_count_inc’; did you
    > > mean ‘g_strv_contains’? [-Wimplicit-function-declaration]
    > > g_ref_count_inc (&data->ref_count);
    > > ^~~~~~~~~~~~~~~
    > > g_strv_contains
    > > ../subprojects/gst-plugins-bad/sys/v4l2codecs/gstv4l2codecvp9dec.c: In
    > > function ‘gst_v4l2_codec_vp9_dec_picture_data_unref’:
    > > ../subprojects/gst-plugins-bad/sys/v4l2codecs/gstv4l2codecvp9dec.c:125:7:
    > > warning: implicit declaration of function ‘g_ref_count_dec’
    > > [-Wimplicit-function-declaration]
    > > if (g_ref_count_dec (&data->ref_count)) {
    > > ^~~~~~~~~~~~~~~
    > > ninja: build stopped: subcommand failed.
    > >
    > > Hope this helps get you started!
    > > Ezequiel
    >
    > Ezequiel and Nicolas,
    >
    > Thanks - I did manage to get gstreamer 1.19.3 built successfully with
    > v4l2codecs finally by getting the correct dependencies. I've attempted
    > to software encode from another system and decode/display on the IMX8M
    > Mini but thus far have not been successful.

    I will post a V2 last today with the Mini's post-processing removed.
    Someone, I apologize that I forget who, mentioned it was fused out of
    the Mini, so the testing I've been doing was with that removed and I
    removed the H1 encoder since the Mini doesn't support JPEG encoding.

    I don't understand the YAML very well, and I'm struggling with the
    dt-bindings which is the main reason I havne't submitted a formal
    patch yet.

    To date, I have only tested with v4l2-compliance and with the fluster
    app that was recommended.

    >
    > I see that v4l2codecs plugin v4l2slh264dec/v4l2slmpeg2dec/v4l2slvp8dec
    > and these all can output video/x-raw NV12/YUY2 which kmssink should
    > accept so I'm attempting the following :
    >
    > # vp8 encode from x86
    > gst-launch-1.0 -v videotestsrc ! video/x-raw,width=800,height=480 !
    > vp8enc ! rtpvp8pay ! udpsink host=172.24.33.15 port=9001
    > # vp8 decode on imx8mm@172.24.33.15 which has a 800x480 display
    > [gst-main] root@focal-venice:~/gstreamer/build# gst-launch-1.0 -v
    > udpsrc port=9001 caps = "application/x-rtp, media=(string)video,
    > clock-rate=(int)90000, encoding-name=(string)VP8, payload=(int)96,
    > ssrc=(uint)2745262155, timestamp-offset=(uint)2515032683,
    > seqnum-offset=(uint)19579, a-framerate=(string)30" ! rtpvp8depay !
    > v4l2slvp8dec ! kmssink
    > Setting pipeline to PAUSED ...
    > Pipeline is live and does not need PREROLL ...
    > /GstPipeline:pipeline0/GstKMSSink:kmssink0: display-width = 800
    > /GstPipeline:pipeline0/GstKMSSink:kmssink0: display-height = 480
    > Pipeline is PREROLLED ...
    > Setting pipeline to PLAYING ...
    > /GstPipeline:pipeline0/GstUDPSrc:udpsrc0.GstPad:src: caps =
    > application/x-rtp, media=(string)video, clock-rate=(int)90000,
    > encoding-name=(string)VP8, payload=(int)96, ssrc=(uint)2745262155,
    > timestamp-offset=(uint)2515032683, seqnum-offset=(uint)19579,
    > a-framerate=(string)30
    > New clock: GstSystemClock
    > /GstPipeline:pipeline0/GstRtpVP8Depay:rtpvp8depay0.GstPad:sink: caps =
    > application/x-rtp, media=(string)video, clock-rate=(int)90000,
    > encoding-name=(string)VP8, payload=(int)96, ssrc=(uint)2745262155,
    > timestamp-offset=(uint)2515032683, seqnum-offset=(uint)19579,
    > a-framerate=(string)30
    > /GstPipeline:pipeline0/GstRtpVP8Depay:rtpvp8depay0.GstPad:src: caps =
    > video/x-vp8, framerate=(fraction)0/1, height=(int)480, width=(int)800,
    > profile=(string)0
    > ERROR: from element /GstPipeline:pipeline0/GstUDPSrc:udpsrc0: Internal
    > data stream error.
    > Additional debug info:
    > ../subprojects/gstreamer/libs/gst/base/gstbasesrc.c(3127):
    > gst_base_src_loop (): /GstPipeline:pipeline0/GstUDPSrc:udpsrc0:
    > streaming stopped, reason not-negotiated (-4)
    > Execution ended after 0:00:02.076839644
    > Setting pipeline to NULL ...
    > Freeing pipeline ...
    >
    > I'm getting the same thing when trying to use h264.
    >
    > I've never quite been able to grasp how to debug GStreamer's
    > negotiation issues. If I end with fakesink it appears to decode so it
    > must be the v4l2slvp8dec to kmssink. I tried forcing the pixel format
    > using 'v4l2slvp8dec ! "video/x-raw,format=(string)NV12" ! kmssink' but
    > I still get the negotiation error.
    >
    > What interrupts should I be seeing in /proc/interrupts? I don't see
    > anything vpu/hantro related there.

    I don't have my Mini with me right now, but with my current patch set
    that I mentioned above, I am able to see the interrupts for the
    video-codec@38300000 increase each time I run a test with fluster.


    >
    > I also want to make sure I have a basic understanding of the vpu
    > drivers and usersapce on the IMX8M Mini. The IMX6Q/DL that I'm more
    > familiar with has a vpu that is supported by the GStreamer video4linux
    > plugin which shows the following (on GStreamer 1.16.2):
    > v4l2jpegenc: V4L2 JPEG Encoder
    > v4l2jpegdec: V4L2 JPEG Decoder
    > v4l2h264enc: V4L2 H.264 Encoder
    > v4l2mpeg4enc: V4L2 MPEG4 Encoder
    > v4l2mpeg4dec: V4L2 MPEG4 Decoder
    > v4l2mpeg2dec: V4L2 MPEG2 Decoder
    > v4l2h264dec: V4L2 H264 Decoder
    > The IMX6Q/DL also has an IPU that has an M2M driver that provides the
    > following for scaling/colorspace conversion:
    > v4l2convert: V4L2 Video Converter
    >
    > I believe what I'm reading is that the IMX8M Mini Hantro codecs are
    > 'stateful' where more software is required to drive them and is
    > supported by the newer v4l2codecs plugin. I haven't been able to
    > understand what kernel version/requirements the v4l2codecs plugin
    > users/requires.
    >
    > I'm also trying to understand how we can get scaling/colorspace
    > conversion on the IMX8M Mini. The IMX8M lacks an IPU... is there some
    > way to utilize scaling/colorspace conversion from the 2D GPU bound to
    > the etnaviv driver?
    >
    > Best regards,
    >
    > Tim

    \
     
     \ /
      Last update: 2021-11-30 21:33    [W:4.078 / U:0.076 seconds]
    ©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site