lkml.org 
[lkml]   [2019]   [May]   [31]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    /
    SubjectRe: [PATCH 10/10] venus: dec: make decoder compliant with stateful codec API
    From
    Date
    Hi,

    On 5/27/19 11:18 AM, Tomasz Figa wrote:
    > On Mon, May 27, 2019 at 4:39 PM Hans Verkuil <hverkuil@xs4all.nl> wrote:
    >>
    >> On 5/27/19 5:51 AM, Tomasz Figa wrote:
    >>> On Tue, May 21, 2019 at 9:27 PM Hans Verkuil <hverkuil@xs4all.nl> wrote:
    >>>>
    >>>> On 5/21/19 11:09 AM, Tomasz Figa wrote:
    >>>>> Hi Stan,
    >>>>>
    >>>>> On Mon, May 20, 2019 at 11:47 PM Stanimir Varbanov
    >>>>> <stanimir.varbanov@linaro.org> wrote:
    >>>>>>
    >>>>>> Hi Tomasz,
    >>>>>>
    >>>>>> On 4/24/19 3:39 PM, Tomasz Figa wrote:
    >>>>>>> On Wed, Apr 24, 2019 at 9:15 PM Stanimir Varbanov
    >>>>>>> <stanimir.varbanov@linaro.org> wrote:
    >>>>>>>>
    >>>>>>>> Hi Hans,
    >>>>>>>>
    >>>>>>>> On 2/15/19 3:44 PM, Hans Verkuil wrote:
    >>>>>>>>> Hi Stanimir,
    >>>>>>>>>
    >>>>>>>>> I never paid much attention to this patch series since others were busy
    >>>>>>>>> discussing it and I had a lot of other things on my plate, but then I heard
    >>>>>>>>> that this patch made G_FMT blocking.
    >>>>>>>>
    >>>>>>>> OK, another option could be to block REQBUF(CAPTURE) until event from hw
    >>>>>>>> is received that the stream is parsed and the resolution is correctly
    >>>>>>>> set by application. Just to note that I'd think to this like a temporal
    >>>>>>>> solution until gstreamer implements v4l events.
    >>>>>>>>
    >>>>>>>> Is that looks good to you?
    >>>>>>>
    >>>>>>> Hmm, I thought we concluded that gstreamer sets the width and height
    >>>>>>> in OUTPUT queue before querying the CAPTURE queue and so making the
    >>>>>>> driver calculate the CAPTURE format based on what's set on OUTPUT
    >>>>>>> would work fine. Did I miss something?
    >>>>>>
    >>>>>> Nobody is miss something.
    >>>>>>
    >>>>>> First some background about how Venus implements stateful codec API.
    >>>>>>
    >>>>>> The Venus firmware can generate two events "sufficient" and
    >>>>>> "insufficient" buffer requirements (this includes decoder output buffer
    >>>>>> size and internal/scratch buffer sizes). Presently I always set minimum
    >>>>>> possible decoder resolution no matter what the user said, and by that
    >>>>>> way I'm sure that "insufficient" event will always be triggered by the
    >>>>>> firmware (the other reason to take this path is because this is the
    >>>>>> least-common-divider for all supported Venus hw/fw versions thus common
    >>>>>> code in the driver). The reconfiguration (during codec Initialization
    >>>>>> sequence) is made from STREAMON(CAPTURE) context. Now, to make that
    >>>>>> re-configuration happen I need to wait for "insufficient" event from
    >>>>>> firmware in order to know the real coded resolution.
    >>>>>>
    >>>>>> In the case of gstreamer where v4l2_events support is missing I have to
    >>>>>> block (wait for firmware event) REQBUF(CAPTURE) (vb2::queue_setup) or
    >>>>>> STREAMON(CAPTURE) (vb2::start_streaming).
    >>>>>>
    >>>>>> I tried to set the coded resolution to the firmware as-is it set by
    >>>>>> gstreamer but then I cannot receive the "sufficient" event for VP8 and
    >>>>>> VP9 codecs. So I return back to the solution with minimum resolution above.
    >>>>>>
    >>>>>> I'm open for suggestions.
    >>>>>
    >>>>> I think you could still keep setting the minimum size and wait for the
    >>>>> "insufficient" event. At the same time, you could speculatively
    >>>>> advertise the expected "sufficient" size on the CAPTURE queue before
    >>>>> the hardware signals those. Even if you mispredict them, you'll get
    >>>>> the event, update the CAPTURE resolution and send the source change
    >>>>> event to the application, which would then give you the correct
    >>>>> buffers. Would that work for you?
    >>>>
    >>>> As I understand it this still would require event support, which gstreamer
    >>>> doesn't have.
    >>>
    >>> I don't think it matches what I remember from the earlier discussion.
    >>> As long as Gstreamer sets the visible resolution (from the container
    >>> AFAIR) on OUTPUT, the driver would adjust it to something that is
    >>> expected to be the right framebuffer resolution and so Gstreamer would
    >>> be able to continue. Of course if the expected value doesn't match, it
    >>> wouldn't work, but it's the same as currently for Coda AFAICT.
    >>>
    >>>>
    >>>> I think it is OK to have REQBUFS sleep in this case. However, I would only
    >>>
    >>> Why REQBUFS? While that could possibly allow us to allocate the right
    >>> buffers, Gstreamer wouldn't be able to know the right format, because
    >>> it would query it before REQBUFS, wouldn't it?
    >>
    >> Oops, you are right. It's got to be in G_FMT(CAPTURE), but *only* if
    >> nobody subscribed to the SOURCE_CHANGE event.
    >>
    >>>
    >>> For this reason, s5p-mfc makes G_FMT(CAPTURE) blocking and if we
    >>> decide to forcefully keep the compatibility, even with in drivers, we
    >>> should probably do the same here.
    >>>
    >>>> enable this behavior if the application didn't subscribe to the SOURCE_CHANGE
    >>>> event. That's easy enough to check in the driver. And that means that if the
    >>>> application is well written, then the driver will behave in a completely
    >>>> standard way that the compliance test can check.
    >>>
    >>> I guess one could have some helpers for this. They would listen to the
    >>> source change events internally and block / wake-up appropriate ioctls
    >>> whenever necessary.
    >>
    >> I really do not want this for new drivers. gstreamer should be fixed.
    >> A blocking G_FMT is just plain bad. Only those drivers that do this, can
    >> still block if nobody subscribed to EVENT_SOURCE_CHANGE.
    >
    > Yeah and that's why I just suggested to mimic coda, which doesn't
    > block, but apparently gstreamer still works with it.

    Unfortunately in Venus case that is not an easy task (as I tried to
    explain why above).

    To have an unified and common code for all different SoCs and
    firmware/hardware versions I decided to set the minimum supported
    resolution for the decoder (no matter what the user said) and trigger
    the reconfiguration event always. Something more, I need the event also
    to retrieve the minimum capture buffers
    (V4L2_CID_MIN_BUFFERS_FOR_CAPTURE) and sizes for capture and
    internal/scratch buffers as well, thus I really need to wait for that
    event.

    So, just to confirm - you are fine with blocking G_FMT (not REQBUF) when
    the user doesn't subscribe for v4l2 events?

    >
    >>
    >>> Another question: If we intend this to be implemented in new drivers
    >>> too, should it be documented in the spec?
    >>
    >> We most certainly do NOT want to implement this in new drivers.
    >>
    >
    > Makes sense.
    >
    > When venus was merged initially, did it already have a blocking G_FMT?

    No it isn't.

    --
    regards,
    Stan

    \
     
     \ /
      Last update: 2019-05-31 10:02    [W:3.954 / U:0.484 seconds]
    ©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site