lkml.org 
[lkml]   [2023]   [Jul]   [21]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    /
    Date
    From
    SubjectRe: Stateless Encoding uAPI Discussion and Proposal
    Hi everyone!

    Just to let you know. I have just pushed a Branch that includes some first
    steps to make the h264-stateless encoder working in Gstreamer. The work is
    based on the VP8 Stateless Encoder patches Benjamin Gaignard created.

    https://gitlab.freedesktop.org/mgrzeschik/gstreamer/-/commits/1.22/topic/h264-stateless-encoder

    The codec this is used with, is the rkvenc that can be found on rockchip
    rk3568. I will send an RFC driver for that in the next weeks after my vacation.

    On Tue, Jul 11, 2023 at 07:12:41PM +0200, Paul Kocialkowski wrote:
    >After various discussions following Andrzej's talk at EOSS, feedback from the
    >Media Summit (which I could not attend unfortunately) and various direct
    >discussions, I have compiled some thoughts and ideas about stateless encoders
    >support with various proposals. This is the result of a few years of interest
    >in the topic, after working on a PoC for the Hantro H1 using the hantro driver,
    >which turned out to have numerous design issues.
    >
    >I am now working on a H.264 encoder driver for Allwinner platforms (currently
    >focusing on the V3/V3s), which already provides some usable bitstream and will
    >be published soon.
    >
    >This is a very long email where I've tried to split things into distinct topics
    >and explain a few concepts to make sure everyone is on the same page.
    >
    ># Bitstream Headers
    >
    >Stateless encoders typically do not generate all the bitstream headers and
    >sometimes no header at all (e.g. Allwinner encoder does not even produce slice
    >headers). There's often some hardware block that makes bit-level writing to the
    >destination buffer easier (deals with alignment, etc).
    >
    >The values of the bitstream headers must be in line with how the compressed
    >data bitstream is generated and generally follow the codec specification.
    >Some encoders might allow configuring all the fields found in the headers,
    >others may only allow configuring a few or have specific constraints regarding
    >which values are allowed.
    >
    >As a result, we cannot expect that any given encoder is able to produce frames
    >for any set of headers. Reporting related constraints and limitations (beyond
    >profile/level) seems quite difficult and error-prone.
    >
    >So it seems that keeping header generation in-kernel only (close to where the
    >hardware is actually configured) is the safest approach.

    For the case with the rkvenc, the headers are also not created by the
    kernel driver. Instead we use the gst_h264_bit_writer_sps/pps functions
    that are part of the codecparsers module.

    ># Codec Features
    >
    >Codecs have many variable features that can be enabled or not and specific
    >configuration fields that can take various values. There is usually some
    >top-level indication of profile/level that restricts what can be used.
    >
    >This is a very similar situation to stateful encoding, where codec-specific
    >controls are used to report and set profile/level and configure these aspects.
    >A particularly nice thing about it is that we can reuse these existing controls
    >and add new ones in the future for features that are not yet covered.
    >
    >This approach feels more flexible than designing new structures with a selected
    >set of parameters (that could match the existing controls) for each codec.

    I back the Idea of generic profiles instead of explicit configuration
    from the usersapace point of view.

    The parameterization works like this:

    Read the sane default parameter set from the driver.
    Modify the parameters based on the userspace decisions.
    - (currently hardcoded and not based on any user input)
    Write the updated parameters back to the driver.

    ># Reference and Reconstruction Management
    <snip>

    ># Frame Types
    <snip>

    ># Rate Control
    <snip>

    ># Regions of Interest
    <snip>

    Since the first shot of the rkvenc is I-Frame only code, these other topics are
    currently undefined and unimplemented in the Gstreamer stack.


    Regards,
    Michael

    --
    Pengutronix e.K. | |
    Steuerwalder Str. 21 | http://www.pengutronix.de/ |
    31137 Hildesheim, Germany | Phone: +49-5121-206917-0 |
    Amtsgericht Hildesheim, HRA 2686 | Fax: +49-5121-206917-5555 |
    [unhandled content-type:application/pgp-signature]
    \
     
     \ /
      Last update: 2023-07-21 20:20    [W:2.793 / U:0.200 seconds]
    ©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site