lkml.org 
[lkml]   [2023]   [Oct]   [20]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    /
    From
    Subject[RFC PATCH v7 00/13] Add audio support in v4l2 framework
    Date
    Audio signal processing also has the requirement for memory to
    memory similar as Video.

    This asrc memory to memory (memory ->asrc->memory) case is a non
    real time use case.

    User fills the input buffer to the asrc module, after conversion, then asrc
    sends back the output buffer to user. So it is not a traditional ALSA playback
    and capture case.

    It is a specific use case, there is no reference in current kernel.
    v4l2 memory to memory is the closed implementation, v4l2 current
    support video, image, radio, tuner, touch devices, so it is not
    complicated to add support for this specific audio case.

    Because we had implemented the "memory -> asrc ->i2s device-> codec"
    use case in ALSA. Now the "memory->asrc->memory" needs
    to reuse the code in asrc driver, so the first 3 patches is for refining
    the code to make it can be shared by the "memory->asrc->memory"
    driver.

    The main change is in the v4l2 side, A /dev/vl4-audioX will be created,
    user applications only use the ioctl of v4l2 framework.

    Other change is to add memory to memory support for two kinds of i.MX ASRC
    module.

    changes in v7:
    - add acked-by from Mark
    - separate commit for fixed point, m2m audio class, audio rate controls
    - use INTEGER_MENU for rate, FIXED_POINT for rate offset
    - remove used fmts
    - address other comments for Hans

    changes in v6:
    - use m2m_prepare/m2m_unprepare/m2m_start/m2m_stop to replace
    m2m_start_part_one/m2m_stop_part_one, m2m_start_part_two/m2m_stop_part_two.
    - change V4L2_CTRL_TYPE_ASRC_RATE to V4L2_CTRL_TYPE_FIXED_POINT
    - fix warning by kernel test rebot
    - remove some unused format V4L2_AUDIO_FMT_XX
    - Get SNDRV_PCM_FORMAT from V4L2_AUDIO_FMT in driver.
    - rename audm2m to viaudm2m.

    changes in v5:
    - remove V4L2_AUDIO_FMT_LPCM
    - define audio pixel format like V4L2_AUDIO_FMT_S8...
    - remove rate and format in struct v4l2_audio_format.
    - Add V4L2_CID_ASRC_SOURCE_RATE and V4L2_CID_ASRC_DEST_RATE controls
    - updata document accordingly.

    changes in v4:
    - update document style
    - separate V4L2_AUDIO_FMT_LPCM and V4L2_CAP_AUDIO_M2M in separate commit

    changes in v3:
    - Modify documents for adding audio m2m support
    - Add audio virtual m2m driver
    - Defined V4L2_AUDIO_FMT_LPCM format type for audio.
    - Defined V4L2_CAP_AUDIO_M2M capability type for audio m2m case.
    - with modification in v4l-utils, pass v4l2-compliance test.

    changes in v2:
    - decouple the implementation in v4l2 and ALSA
    - implement the memory to memory driver as a platfrom driver
    and move it to driver/media
    - move fsl_asrc_common.h to include/sound folder

    Shengjiu Wang (13):
    ASoC: fsl_asrc: define functions for memory to memory usage
    ASoC: fsl_easrc: define functions for memory to memory usage
    ASoC: fsl_asrc: move fsl_asrc_common.h to include/sound
    ASoC: fsl_asrc: register m2m platform device
    ASoC: fsl_easrc: register m2m platform device
    media: uapi: Add V4L2_CAP_AUDIO_M2M capability flag
    media: v4l2: Add audio capture and output support
    media: uapi: define audio sample format fourcc type
    media: uapi: Add V4L2_CTRL_CLASS_M2M_AUDIO
    media: uapi: Add V4L2_CTRL_TYPE_FIXED_POINT
    media: uapi: Add audio rate controls support
    media: imx-asrc: Add memory to memory driver
    media: vim2m_audio: add virtual driver for audio memory to memory

    .../userspace-api/media/v4l/buffer.rst | 6 +
    .../userspace-api/media/v4l/common.rst | 1 +
    .../media/v4l/dev-audio-mem2mem.rst | 71 +
    .../userspace-api/media/v4l/devices.rst | 1 +
    .../media/v4l/ext-ctrls-audio-m2m.rst | 39 +
    .../userspace-api/media/v4l/pixfmt-audio.rst | 87 ++
    .../userspace-api/media/v4l/pixfmt.rst | 1 +
    .../media/v4l/vidioc-enum-fmt.rst | 2 +
    .../media/v4l/vidioc-g-ext-ctrls.rst | 4 +
    .../userspace-api/media/v4l/vidioc-g-fmt.rst | 4 +
    .../media/v4l/vidioc-querycap.rst | 3 +
    .../media/v4l/vidioc-queryctrl.rst | 6 +
    .../media/videodev2.h.rst.exceptions | 4 +
    .../media/common/videobuf2/videobuf2-v4l2.c | 4 +
    drivers/media/platform/nxp/Kconfig | 12 +
    drivers/media/platform/nxp/Makefile | 1 +
    drivers/media/platform/nxp/imx-asrc.c | 1207 +++++++++++++++++
    drivers/media/test-drivers/Kconfig | 9 +
    drivers/media/test-drivers/Makefile | 1 +
    drivers/media/test-drivers/vim2m_audio.c | 680 ++++++++++
    drivers/media/v4l2-core/v4l2-ctrls-api.c | 5 +-
    drivers/media/v4l2-core/v4l2-ctrls-core.c | 2 +
    drivers/media/v4l2-core/v4l2-ctrls-defs.c | 16 +
    drivers/media/v4l2-core/v4l2-dev.c | 17 +
    drivers/media/v4l2-core/v4l2-ioctl.c | 66 +
    include/media/v4l2-dev.h | 2 +
    include/media/v4l2-ioctl.h | 34 +
    .../fsl => include/sound}/fsl_asrc_common.h | 60 +
    include/uapi/linux/v4l2-controls.h | 9 +
    include/uapi/linux/videodev2.h | 42 +
    sound/soc/fsl/fsl_asrc.c | 144 ++
    sound/soc/fsl/fsl_asrc.h | 4 +-
    sound/soc/fsl/fsl_asrc_dma.c | 2 +-
    sound/soc/fsl/fsl_easrc.c | 233 ++++
    sound/soc/fsl/fsl_easrc.h | 6 +-
    35 files changed, 2781 insertions(+), 4 deletions(-)
    create mode 100644 Documentation/userspace-api/media/v4l/dev-audio-mem2mem.rst
    create mode 100644 Documentation/userspace-api/media/v4l/ext-ctrls-audio-m2m.rst
    create mode 100644 Documentation/userspace-api/media/v4l/pixfmt-audio.rst
    create mode 100644 drivers/media/platform/nxp/imx-asrc.c
    create mode 100644 drivers/media/test-drivers/vim2m_audio.c
    rename {sound/soc/fsl => include/sound}/fsl_asrc_common.h (60%)

    --
    2.34.1

    \
     
     \ /
      Last update: 2023-10-20 12:11    [W:6.206 / U:0.016 seconds]
    ©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site