lkml.org 
[lkml]   [2023]   [Jul]   [24]   [last100]   RSS Feed
Views: [wrap][no wrap]   [headers]  [forward] 
 
Messages in this thread
    /
    Date
    SubjectRe: [PATCH v4 00/32] Introduce QC USB SND audio offloading support
    From
    Apologies for the long delay in addressing some of your comments and 
    feedback. I understand it is difficult to refresh a lot of the material
    after such a long gap, but hopefully I have addressed some of the
    concerns in the latest revision. Appreciate all the suggestions/reviews!

    Thanks
    Wesley Cheng

    On 7/24/2023 7:33 PM, Wesley Cheng wrote:
    > Several Qualcomm based chipsets can support USB audio offloading to a
    > dedicated audio DSP, which can take over issuing transfers to the USB
    > host controller. The intention is to reduce the load on the main
    > processors in the SoC, and allow them to be placed into lower power modes.
    > There are several parts to this design:
    > 1. Adding ASoC binding layer
    > 2. Create a USB backend for Q6DSP
    > 3. Introduce XHCI interrupter support
    > 4. Create vendor ops for the USB SND driver
    >
    > USB | ASoC
    > --------------------------------------------------------------------
    > | _________________________
    > | |sm8250 platform card |
    > | |_________________________|
    > | | |
    > | ___V____ ____V____
    > | |Q6USB | |Q6AFE |
    > | |"codec" | |"cpu" |
    > | |________| |_________|
    > | ^ ^ ^
    > | | |________|
    > | ___V____ |
    > | |SOC-USB | |
    > ________ ________ | | |
    > |USB SND |<--->|QC offld|<------------>|________| |
    > |(card.c)| | |<---------- |
    > |________| |________|___ | | |
    > ^ ^ | | | ____________V_________
    > | | | | | |APR/GLINK |
    > __ V_______________V_____ | | | |______________________|
    > |USB SND (endpoint.c) | | | | ^
    > |_________________________| | | | |
    > ^ | | | ___________V___________
    > | | | |->|audio DSP |
    > ___________V_____________ | | |_______________________|
    > |XHCI HCD |<- |
    > |_________________________| |
    >
    >
    > Adding ASoC binding layer:
    > soc-usb: Intention is to treat a USB port similar to a headphone jack.
    > The port is always present on the device, but cable/pin status can be
    > enabled/disabled. Expose mechanisms for USB backend ASoC drivers to
    > communicate with USB SND.
    >
    > Create a USB backend for Q6DSP:
    > q6usb: Basic backend driver that will be responsible for maintaining the
    > resources needed to initiate a playback stream using the Q6DSP. Will
    > be the entity that checks to make sure the connected USB audio device
    > supports the requested PCM format. If it does not, the PCM open call will
    > fail, and userpsace ALSA can take action accordingly.
    >
    > Introduce XHCI interrupter support:
    > XHCI HCD supports multiple interrupters, which allows for events to be routed
    > to different event rings. This is determined by "Interrupter Target" field
    > specified in Section "6.4.1.1 Normal TRB" of the XHCI specification.
    >
    > Events in the offloading case will be routed to an event ring that is assigned
    > to the audio DSP.
    >
    > Create vendor ops for the USB SND driver:
    > qc_audio_offload: This particular driver has several components associated
    > with it:
    > - QMI stream request handler
    > - XHCI interrupter and resource management
    > - audio DSP memory management
    >
    > When the audio DSP wants to enable a playback stream, the request is first
    > received by the ASoC platform sound card. Depending on the selected route,
    > ASoC will bring up the individual DAIs in the path. The Q6USB backend DAI
    > will send an AFE port start command (with enabling the USB playback path), and
    > the audio DSP will handle the request accordingly.
    >
    > Part of the AFE USB port start handling will have an exchange of control
    > messages using the QMI protocol. The qc_audio_offload driver will populate the
    > buffer information:
    > - Event ring base address
    > - EP transfer ring base address
    >
    > and pass it along to the audio DSP. All endpoint management will now be handed
    > over to the DSP, and the main processor is not involved in transfers.
    >
    > Overall, implementing this feature will still expose separate sound card and PCM
    > devices for both the platorm card and USB audio device:
    > 0 [SM8250MTPWCD938]: sm8250 - SM8250-MTP-WCD9380-WSA8810-VA-D
    > SM8250-MTP-WCD9380-WSA8810-VA-DMIC
    > 1 [Audio ]: USB-Audio - USB Audio
    > Generic USB Audio at usb-xhci-hcd.1.auto-1.4, high speed
    >
    > This is to ensure that userspace ALSA entities can decide which route to take
    > when executing the audio playback. In the above, if card#1 is selected, then
    > USB audio data will take the legacy path over the USB PCM drivers, etc...
    >
    > This feature was validated using:
    > - tinymix: set/enable the multimedia path to route to USB backend
    > - tinyplay: issue playback on platform card
    >
    > Changelog
    > --------------------------------------------
    > Changes in v4:
    > - Rebased to xhci/for-usb-next
    > - Addressed some dt-bindings comments
    >
    > XHCI:
    > - Pulled in latest changes from Mathias' feature_interrupters branch:
    > https://git.kernel.org/pub/scm/linux/kernel/git/mnyman/xhci.git/log/?h=feature_interrupters
    >
    > - Fixed commit text and signage for the XHCI sideband/interrupter related changes
    > - Added some logic to address the FIXME tags mentioned throughout the commits, such
    > as handling multi segment rings and building the SGT, locking concerns, and ep
    > cleanup operations.
    > - Removed some fixme tags for conditions that may not be needed/addressed.
    > - Repurposed the new endpoint stop sync API to be utilized in other places.
    > - Fixed potential compile issue if XHCI sideband config is not defined.
    >
    > ASoC:
    > - Added sound jack control into the Q6USB driver. Allows for userpsace to know when
    > an offload capable device is connected.
    >
    > USB SND:
    > - Avoided exporting _snd_pcm_hw_param_set based on Takashi's recommendation.
    > - Split USB QMI packet header definitions into a separate commit. This is used to
    > properly allow the QMI interface driver to parse and route QMI packets accordingly
    > - Added a "depends on" entry when enabling QC audio offload to avoid compile time
    > issues.
    >
    > Changes in v3:
    > - Changed prefix from RFC to PATCH
    > - Rebased entire series to usb-next
    > - Updated copyright years
    >
    > XHCI:
    > - Rebased changes on top of XHCI changes merged into usb-next, and only added
    > changes that were still under discussion.
    > - Added change to read in the "num-hc-interrupters" device property.
    >
    > ASoC:
    > - qusb6 USB backend
    > - Incorporated suggestions to fetch iommu information with existing APIs
    > - Added two new sound kcontrols to fetch offload status and offload device
    > selection.
    > - offload status - will return the card and pcm device in use
    > tinymix -D 0 get 1 --> 1, 0 (offload in progress on card#1 pcm#0)
    >
    > - device selection - set the card and pcm device to enable offload on. Ex.:
    > tinymix -D 0 set 1 2 0 --> sets offload on card#2 pcm#0
    > (this should be the USB card)
    >
    > USB SND:
    > - Fixed up some locking related concerns for registering platform ops.
    > - Moved callbacks under the register_mutex, so that
    > - Modified APIs to properly pass more information about the USB SND device, so
    > that the Q6USB backend can build a device list/map, in order to monitor offload
    > status and device selection.
    >
    > Changes in v2:
    >
    > XHCI:
    > - Replaced XHCI and HCD changes with Mathias' XHCI interrupter changes
    > in his tree:
    > https://git.kernel.org/pub/scm/linux/kernel/git/mnyman/xhci.git/log/?h=feature_interrupters
    >
    > Adjustments made to Mathias' changes:
    > - Created xhci-intr.h to export/expose interrupter APIs versus exposing xhci.h.
    > Moved dependent structures to this file as well. (so clients can parse out
    > information from "struct xhci_interrupter")
    > - Added some basic locking when requesting interrupters.
    > - Fixed up some sanity checks.
    > - Removed clearing of the ERSTBA during freeing of the interrupter. (pending
    > issue where SMMU fault occurs if DMA addr returned is 64b - TODO)
    >
    > - Clean up pending events in the XHCI secondary interrupter. While testing USB
    > bus suspend, it was seen that on bus resume, the xHCI HC would run into a command
    > timeout.
    > - Added offloading APIs to xHCI to fetch transfer and event ring information.
    >
    > ASoC:
    > - Modified soc-usb to allow for multiple USB port additions. For this to work,
    > the USB offload driver has to have a reference to the USB backend by adding
    > a "usb-soc-be" DT entry to the device saved into XHCI sysdev.
    > - Created separate dt-bindings for defining USB_RX port.
    > - Increased APR timeout to accommodate the situation where the AFE port start
    > command could be delayed due to having to issue a USB bus resume while
    > handling the QMI stream start command.
    >
    > USB SND:
    > - Added a platform ops during usb_audio_suspend(). This allows for the USB
    > offload driver to halt the audio stream when system enters PM suspend. This
    > ensures the audio DSP is not issuing transfers on the USB bus.
    > - Do not override platform ops if they are already populated.
    > - Introduce a shared status variable between the USB offload and USB SND layers,
    > to ensure that only one path is active at a time. If the USB bus is occupied,
    > then userspace is notified that the path is busy.
    >
    > Mathias Nyman (3):
    > xhci: add support to allocate several interrupters
    > xhci: add helper to stop endpoint and wait for completion
    > xhci: sideband: add initial api to register a sideband entity
    >
    > Wesley Cheng (29):
    > usb: host: xhci-mem: Cleanup pending secondary event ring events
    > usb: host: xhci-mem: Allow for interrupter clients to choose specific
    > index
    > ASoC: Add SOC USB APIs for adding an USB backend
    > ASoC: dt-bindings: qcom,q6dsp-lpass-ports: Add USB_RX port
    > ASoC: qcom: qdsp6: Introduce USB AFE port to q6dsp
    > ASoC: qdsp6: q6afe: Increase APR timeout
    > ASoC: qcom: Add USB backend ASoC driver for Q6
    > sound: usb: card: Introduce USB SND platform op callbacks
    > sound: usb: Export USB SND APIs for modules
    > dt-bindings: usb: dwc3: Add snps,num-hc-interrupters definition
    > usb: dwc3: Add DT parameter to specify maximum number of interrupters
    > usb: host: xhci-plat: Set XHCI max interrupters if property is present
    > sound: usb: pcm: Export fixed rate check USB SND API
    > sound: usb: qcom: Add USB QMI definitions
    > sound: usb: Introduce QC USB SND offloading support
    > sound: usb: card: Check for support for requested audio format
    > sound: soc: soc-usb: Add PCM format check API for USB backend
    > sound: soc: qcom: qusb6: Ensure PCM format is supported by USB audio
    > device
    > sound: usb: Prevent starting of audio stream if in use
    > ASoC: dt-bindings: Add Q6USB backend bindings
    > ASoC: dt-bindings: Update example for enabling USB offload on SM8250
    > ASoC: qcom: qdsp6: q6afe: Split USB AFE dev_token param into separate
    > API
    > sound: Pass USB SND card and PCM information to SOC USB
    > sound: soc: qdsp6: Add SND kcontrol to select offload device
    > sound: soc: qdsp6: Add SND kcontrol for fetching offload status
    > sound: soc: qcom: q6usb: Add headphone jack for offload connection
    > status
    > sound: usb: qc_audio_offload: Use card and PCM index from QMI request
    > sound: usb: card: Allow for rediscovery of connected USB SND devices
    > sound: soc: soc-usb: Rediscover USB SND devices on USB port add
    >
    > .../bindings/sound/qcom,q6usb-dais.yaml | 49 +
    > .../bindings/sound/qcom,sm8250.yaml | 15 +
    > .../devicetree/bindings/usb/snps,dwc3.yaml | 13 +
    > arch/arm64/boot/dts/qcom/sm8350.dtsi.rej | 14 +
    > arch/arm64/configs/defconfig.rej | 24 +
    > drivers/usb/dwc3/core.c | 12 +
    > drivers/usb/dwc3/core.h | 2 +
    > drivers/usb/dwc3/host.c | 5 +-
    > drivers/usb/host/Kconfig | 9 +
    > drivers/usb/host/Makefile | 4 +
    > drivers/usb/host/xhci-debugfs.c | 2 +-
    > drivers/usb/host/xhci-hub.c | 29 +-
    > drivers/usb/host/xhci-mem.c | 167 +-
    > drivers/usb/host/xhci-plat.c | 2 +
    > drivers/usb/host/xhci-ring.c | 2 +-
    > drivers/usb/host/xhci-sideband.c | 195 ++
    > drivers/usb/host/xhci.c | 109 +-
    > drivers/usb/host/xhci.h | 102 +-
    > .../sound/qcom,q6dsp-lpass-ports.h | 1 +
    > include/linux/usb/xhci-intr.h | 86 +
    > include/linux/usb/xhci-sideband.h | 90 +
    > include/sound/q6usboffload.h | 20 +
    > include/sound/soc-usb.h | 39 +
    > sound/core/oss/pcm_oss.c | 4 +-
    > sound/soc/Makefile | 2 +-
    > sound/soc/qcom/Kconfig | 4 +
    > sound/soc/qcom/qdsp6/Makefile | 1 +
    > sound/soc/qcom/qdsp6/q6afe-dai.c | 50 +
    > sound/soc/qcom/qdsp6/q6afe.c | 208 +-
    > sound/soc/qcom/qdsp6/q6afe.h | 47 +-
    > sound/soc/qcom/qdsp6/q6dsp-lpass-ports.c | 23 +
    > sound/soc/qcom/qdsp6/q6dsp-lpass-ports.h | 1 +
    > sound/soc/qcom/qdsp6/q6routing.c | 9 +
    > sound/soc/qcom/qdsp6/q6usb.c | 458 +++++
    > sound/soc/soc-usb.c | 197 ++
    > sound/usb/Kconfig | 15 +
    > sound/usb/Makefile | 2 +-
    > sound/usb/card.c | 85 +
    > sound/usb/card.h | 32 +
    > sound/usb/endpoint.c | 2 +
    > sound/usb/helper.c | 1 +
    > sound/usb/pcm.c | 29 +-
    > sound/usb/pcm.h | 12 +
    > sound/usb/qcom/Makefile | 2 +
    > sound/usb/qcom/qc_audio_offload.c | 1820 +++++++++++++++++
    > sound/usb/qcom/usb_audio_qmi_v01.c | 892 ++++++++
    > sound/usb/qcom/usb_audio_qmi_v01.h | 162 ++
    > 47 files changed, 4870 insertions(+), 179 deletions(-)
    > create mode 100644 Documentation/devicetree/bindings/sound/qcom,q6usb-dais.yaml
    > create mode 100644 arch/arm64/boot/dts/qcom/sm8350.dtsi.rej
    > create mode 100644 arch/arm64/configs/defconfig.rej
    > create mode 100644 drivers/usb/host/xhci-sideband.c
    > create mode 100644 include/linux/usb/xhci-intr.h
    > create mode 100644 include/linux/usb/xhci-sideband.h
    > create mode 100644 include/sound/q6usboffload.h
    > create mode 100644 include/sound/soc-usb.h
    > create mode 100644 sound/soc/qcom/qdsp6/q6usb.c
    > create mode 100644 sound/soc/soc-usb.c
    > create mode 100644 sound/usb/qcom/Makefile
    > create mode 100644 sound/usb/qcom/qc_audio_offload.c
    > create mode 100644 sound/usb/qcom/usb_audio_qmi_v01.c
    > create mode 100644 sound/usb/qcom/usb_audio_qmi_v01.h
    >

    \
     
     \ /
      Last update: 2023-07-25 04:44    [W:5.255 / U:0.208 seconds]
    ©2003-2020 Jasper Spaans|hosted at Digital Ocean and TransIP|Read the blog|Advertise on this site