From patchwork Fri Mar 26 06:44:00 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: You-Sheng Yang X-Patchwork-Id: 1458655 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (no SPF record) smtp.mailfrom=lists.ubuntu.com (client-ip=91.189.94.19; helo=huckleberry.canonical.com; envelope-from=kernel-team-bounces@lists.ubuntu.com; receiver=) Received: from huckleberry.canonical.com (huckleberry.canonical.com [91.189.94.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4F6CBH1LXQz9sS8; Fri, 26 Mar 2021 17:44:22 +1100 (AEDT) Received: from localhost ([127.0.0.1] helo=huckleberry.canonical.com) by huckleberry.canonical.com with esmtp (Exim 4.86_2) (envelope-from ) id 1lPgCd-0006tU-4u; Fri, 26 Mar 2021 06:44:15 +0000 Received: from mail-pf1-f170.google.com ([209.85.210.170]) by huckleberry.canonical.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1lPgCa-0006tG-Je for kernel-team@lists.ubuntu.com; Fri, 26 Mar 2021 06:44:12 +0000 Received: by mail-pf1-f170.google.com with SMTP id y200so4227048pfb.5 for ; Thu, 25 Mar 2021 23:44:12 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:subject:date:message-id:mime-version :content-transfer-encoding; bh=rH4vaWItR9jYqwOnrDs9ChEIF4BsuocqWtRWAepVLEg=; b=fNIs74a85VedF+pxHWpyL83h3BX0KYiBdTICKa5tXRwrCw+zL2WGcB3FdjocT68QJT UT7oR5R/yKY+QhmBgwT77loAnlZW4iwHKRhJkQBHWtR/8hIbFCnZrQMdG4h8Ph/Hma2E 8BE0WUWYSouoEUthcvNOIE5ECzMcapsAO6YSG8tA0dtWWSakx/eJmQ6Te9rs79neamcX CVTEP9DEIDjthJZDUrRfl7AHcQUhcD6+WE7tJyFroMYg61lU+eR9cgOShnUFLmMizmdD RHwFk7fZH9FG0h+/eq7EJmgjuSAIpQ3srEa2S3mcXK8H5gjjMHJH9s9pT9h3NpQtQ7Zd lfWQ== X-Gm-Message-State: AOAM5338GyMWigAENykl5HY2lmqBmFgQKBCjbXWR2nFtr/KZLkxsRvEj 7lg/dpqZjSkCwgEXoGJf09D3JyPe8Hc= X-Google-Smtp-Source: ABdhPJxM7NfLsivntmIA/JGQkV1BQFApqY11x08RBA/77scFxRs2KpSowyCXJawaJzK4OljjJlsCMw== X-Received: by 2002:a05:6a00:2389:b029:21a:d3b4:e5 with SMTP id f9-20020a056a002389b029021ad3b400e5mr11745784pfc.39.1616741049817; Thu, 25 Mar 2021 23:44:09 -0700 (PDT) Received: from localhost (61-220-137-37.HINET-IP.hinet.net. [61.220.137.37]) by smtp.gmail.com with ESMTPSA id 38sm7693061pgk.30.2021.03.25.23.44.08 for (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Thu, 25 Mar 2021 23:44:08 -0700 (PDT) From: You-Sheng Yang To: kernel-team@lists.ubuntu.com Subject: [PATCH 0/7][SRU][OEM-5.10] Add Intel IPU6 driver Date: Fri, 26 Mar 2021 14:44:00 +0800 Message-Id: <20210326064407.1789434-1-vicamo.yang@canonical.com> X-Mailer: git-send-email 2.30.2 MIME-Version: 1.0 Received-SPF: pass client-ip=209.85.210.170; envelope-from=vicamo@gmail.com; helo=mail-pf1-f170.google.com X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.20 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: "kernel-team" BugLink: https://bugs.launchpad.net/bugs/1921345 [Impact] There are new platforms with MIPI camera based on Intel IPU6(Imaging Processing Unit version 6) currently unsupported in Ubuntu. [Fix] Intel IPU6 can be enabled by integration of following kernel driver, firmware, userspace HAL libraries as well as a GStreamer source element: * https://github.com/intel/ipu6-drivers * https://github.com/intel/ipu6-camera-bins * https://github.com/intel/ipu6-camera-hal * https://github.com/intel/icamerasrc This patchset took kernel patches from intel/ipu6-drivers. [Test Case] With kernel/firmware properly installed, the hardware should be ready for further development: $ dmesg | grep ipu intel-ipu6 intel-ipu: enabling device (0000 -> 0002) intel-ipu6 intel-ipu: Device 0x9a19 (rev: 0x1) intel-ipu6 intel-ipu: physical base address 0x6054000000 intel-ipu6 intel-ipu: mapped as: 0x0000000097793328 intel-ipu6 intel-ipu: IPU in secure mode intel-ipu6 intel-ipu: IPC reset done intel-ipu6 intel-ipu: cpd file name: intel/ipu6_fw.bin intel-ipu6 intel-ipu: FW version: 20201222 intel-ipu6 intel-ipu: Sending BOOT_LOAD to CSE intel-ipu6 intel-ipu: Sending AUTHENTICATE_RUN to CSE intel-ipu6 intel-ipu: CSE authenticate_run done intel-ipu6 intel-ipu: IPU driver version 1.0 intel-ipu6-psys intel-ipu6-psys0: pkg_dir entry count:8 intel-ipu6-psys intel-ipu6-psys0: psys probe minor: 0 intel-ipu6-isys intel-ipu6-isys0: bind ov01a1s 20-0036 nlanes is 1 port is 1 intel-ipu6-isys intel-ipu6-isys0: All sensor registration completed. intel-ipu6-isys intel-ipu6-isys0: stream on ov01a1s 20-0036 There should be nearly 24 video4linux devices created under /dev. [Where problems could occur] MIPI camera through Intel IPU6 takes also firmware blobs loaded in runtime from user space, addtional softwares are also required to fully enable it for oridinary use. The exposed video devices are hidden from general users and a relay daemon+v4l2loopback is used to enable use from existing applications. From kernel's point of view, there is still a known issue that it takes root priviledge to access these devices. [Other Info] So far Intel has no plan to commit this driver to upstream yet, so it's only nominated for oem-5.10. It will only be nominated to generic kernels when the plan/requirements have changed. The original driver Kconfig would disable VIDEO_IPU3_CIO2, but that's reverted to avoid unnecessary changes to existing users. Wang Yating (6): intel ipu drivers first release IPU driver release WW48 IPU driver release WW48 with MCU IPU driver release WW52 IPU driver release WW04 UBUNTU: SAUCE: integrate IPU6 builds You-Sheng Yang (1): UBUNTU: [Config] updateconfigs for IPU6 driver debian.oem/config/config.common.ubuntu | 6 + drivers/media/i2c/Kconfig | 33 + drivers/media/i2c/Makefile | 4 + drivers/media/i2c/hm11b1.c | 1366 +++++++++++++ drivers/media/i2c/ov01a1s.c | 935 +++++++++ drivers/media/i2c/pmic_dsc1.c | 158 ++ drivers/media/i2c/pmic_dsc1.h | 33 + drivers/media/pci/Kconfig | 2 +- drivers/media/pci/intel/Kconfig | 32 + drivers/media/pci/intel/Makefile | 11 +- drivers/media/pci/intel/ipu-bus.c | 255 +++ drivers/media/pci/intel/ipu-bus.h | 67 + drivers/media/pci/intel/ipu-buttress.c | 1399 +++++++++++++ drivers/media/pci/intel/ipu-buttress.h | 129 ++ drivers/media/pci/intel/ipu-cpd.c | 465 +++++ drivers/media/pci/intel/ipu-cpd.h | 110 + drivers/media/pci/intel/ipu-dma.c | 409 ++++ drivers/media/pci/intel/ipu-dma.h | 19 + drivers/media/pci/intel/ipu-fw-com.c | 496 +++++ drivers/media/pci/intel/ipu-fw-com.h | 48 + drivers/media/pci/intel/ipu-fw-isys.c | 700 +++++++ drivers/media/pci/intel/ipu-fw-isys.h | 826 ++++++++ drivers/media/pci/intel/ipu-fw-psys.c | 433 ++++ drivers/media/pci/intel/ipu-fw-psys.h | 394 ++++ .../media/pci/intel/ipu-isys-csi2-be-soc.c | 293 +++ drivers/media/pci/intel/ipu-isys-csi2-be.c | 325 +++ drivers/media/pci/intel/ipu-isys-csi2-be.h | 66 + drivers/media/pci/intel/ipu-isys-csi2.c | 662 ++++++ drivers/media/pci/intel/ipu-isys-csi2.h | 164 ++ drivers/media/pci/intel/ipu-isys-media.h | 77 + drivers/media/pci/intel/ipu-isys-queue.c | 1062 ++++++++++ drivers/media/pci/intel/ipu-isys-queue.h | 142 ++ drivers/media/pci/intel/ipu-isys-subdev.c | 657 ++++++ drivers/media/pci/intel/ipu-isys-subdev.h | 153 ++ drivers/media/pci/intel/ipu-isys-tpg.c | 311 +++ drivers/media/pci/intel/ipu-isys-tpg.h | 99 + drivers/media/pci/intel/ipu-isys-video.c | 1768 +++++++++++++++++ drivers/media/pci/intel/ipu-isys-video.h | 181 ++ drivers/media/pci/intel/ipu-isys.c | 1473 ++++++++++++++ drivers/media/pci/intel/ipu-isys.h | 237 +++ drivers/media/pci/intel/ipu-mmu.c | 787 ++++++++ drivers/media/pci/intel/ipu-mmu.h | 67 + drivers/media/pci/intel/ipu-pdata.h | 255 +++ drivers/media/pci/intel/ipu-psys-compat32.c | 226 +++ drivers/media/pci/intel/ipu-psys.c | 1617 +++++++++++++++ drivers/media/pci/intel/ipu-psys.h | 218 ++ drivers/media/pci/intel/ipu-trace.c | 882 ++++++++ drivers/media/pci/intel/ipu-trace.h | 312 +++ drivers/media/pci/intel/ipu.c | 813 ++++++++ drivers/media/pci/intel/ipu.h | 106 + drivers/media/pci/intel/ipu6/Makefile | 62 + .../media/pci/intel/ipu6/ipu-fw-resources.c | 103 + .../intel/ipu6/ipu-platform-buttress-regs.h | 317 +++ .../intel/ipu6/ipu-platform-isys-csi2-reg.h | 277 +++ .../media/pci/intel/ipu6/ipu-platform-isys.h | 26 + .../media/pci/intel/ipu6/ipu-platform-psys.h | 78 + .../media/pci/intel/ipu6/ipu-platform-regs.h | 333 ++++ .../pci/intel/ipu6/ipu-platform-resources.h | 102 + drivers/media/pci/intel/ipu6/ipu-platform.h | 33 + drivers/media/pci/intel/ipu6/ipu-resources.c | 861 ++++++++ .../media/pci/intel/ipu6/ipu6-fw-resources.c | 582 ++++++ drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c | 513 +++++ drivers/media/pci/intel/ipu6/ipu6-isys-csi2.h | 14 + drivers/media/pci/intel/ipu6/ipu6-isys-gpc.c | 205 ++ drivers/media/pci/intel/ipu6/ipu6-isys-phy.c | 595 ++++++ drivers/media/pci/intel/ipu6/ipu6-isys-phy.h | 159 ++ drivers/media/pci/intel/ipu6/ipu6-isys.c | 322 +++ .../media/pci/intel/ipu6/ipu6-l-scheduler.c | 611 ++++++ .../pci/intel/ipu6/ipu6-platform-resources.h | 197 ++ drivers/media/pci/intel/ipu6/ipu6-ppg.c | 559 ++++++ drivers/media/pci/intel/ipu6/ipu6-ppg.h | 38 + drivers/media/pci/intel/ipu6/ipu6-psys-gpc.c | 212 ++ drivers/media/pci/intel/ipu6/ipu6-psys.c | 1056 ++++++++++ drivers/media/pci/intel/ipu6/ipu6.c | 368 ++++ .../pci/intel/ipu6/ipu6se-fw-resources.c | 352 ++++ .../intel/ipu6/ipu6se-platform-resources.h | 127 ++ drivers/usb/Kconfig | 2 + drivers/usb/Makefile | 2 + drivers/usb/intel_ulpss/Kconfig | 11 + drivers/usb/intel_ulpss/Makefile | 3 + drivers/usb/intel_ulpss/diag_stub.c | 116 ++ drivers/usb/intel_ulpss/diag_stub.h | 19 + drivers/usb/intel_ulpss/gpio_stub.c | 459 +++++ drivers/usb/intel_ulpss/gpio_stub.h | 13 + drivers/usb/intel_ulpss/i2c_stub.c | 456 +++++ drivers/usb/intel_ulpss/i2c_stub.h | 21 + drivers/usb/intel_ulpss/mng_stub.c | 244 +++ drivers/usb/intel_ulpss/mng_stub.h | 18 + .../usb/intel_ulpss/protocol_intel_ulpss.h | 173 ++ drivers/usb/intel_ulpss/ulpss_bridge.c | 529 +++++ drivers/usb/intel_ulpss/ulpss_bridge.h | 77 + drivers/usb/intel_ulpss/usb_stub.c | 285 +++ drivers/usb/intel_ulpss/usb_stub.h | 49 + include/media/ipu-isys.h | 44 + include/uapi/linux/ipu-isys.h | 15 + include/uapi/linux/ipu-psys.h | 121 ++ 96 files changed, 32040 insertions(+), 2 deletions(-) create mode 100644 drivers/media/i2c/hm11b1.c create mode 100644 drivers/media/i2c/ov01a1s.c create mode 100644 drivers/media/i2c/pmic_dsc1.c create mode 100644 drivers/media/i2c/pmic_dsc1.h create mode 100644 drivers/media/pci/intel/Kconfig create mode 100644 drivers/media/pci/intel/ipu-bus.c create mode 100644 drivers/media/pci/intel/ipu-bus.h create mode 100644 drivers/media/pci/intel/ipu-buttress.c create mode 100644 drivers/media/pci/intel/ipu-buttress.h create mode 100644 drivers/media/pci/intel/ipu-cpd.c create mode 100644 drivers/media/pci/intel/ipu-cpd.h create mode 100644 drivers/media/pci/intel/ipu-dma.c create mode 100644 drivers/media/pci/intel/ipu-dma.h create mode 100644 drivers/media/pci/intel/ipu-fw-com.c create mode 100644 drivers/media/pci/intel/ipu-fw-com.h create mode 100644 drivers/media/pci/intel/ipu-fw-isys.c create mode 100644 drivers/media/pci/intel/ipu-fw-isys.h create mode 100644 drivers/media/pci/intel/ipu-fw-psys.c create mode 100644 drivers/media/pci/intel/ipu-fw-psys.h create mode 100644 drivers/media/pci/intel/ipu-isys-csi2-be-soc.c create mode 100644 drivers/media/pci/intel/ipu-isys-csi2-be.c create mode 100644 drivers/media/pci/intel/ipu-isys-csi2-be.h create mode 100644 drivers/media/pci/intel/ipu-isys-csi2.c create mode 100644 drivers/media/pci/intel/ipu-isys-csi2.h create mode 100644 drivers/media/pci/intel/ipu-isys-media.h create mode 100644 drivers/media/pci/intel/ipu-isys-queue.c create mode 100644 drivers/media/pci/intel/ipu-isys-queue.h create mode 100644 drivers/media/pci/intel/ipu-isys-subdev.c create mode 100644 drivers/media/pci/intel/ipu-isys-subdev.h create mode 100644 drivers/media/pci/intel/ipu-isys-tpg.c create mode 100644 drivers/media/pci/intel/ipu-isys-tpg.h create mode 100644 drivers/media/pci/intel/ipu-isys-video.c create mode 100644 drivers/media/pci/intel/ipu-isys-video.h create mode 100644 drivers/media/pci/intel/ipu-isys.c create mode 100644 drivers/media/pci/intel/ipu-isys.h create mode 100644 drivers/media/pci/intel/ipu-mmu.c create mode 100644 drivers/media/pci/intel/ipu-mmu.h create mode 100644 drivers/media/pci/intel/ipu-pdata.h create mode 100644 drivers/media/pci/intel/ipu-psys-compat32.c create mode 100644 drivers/media/pci/intel/ipu-psys.c create mode 100644 drivers/media/pci/intel/ipu-psys.h create mode 100644 drivers/media/pci/intel/ipu-trace.c create mode 100644 drivers/media/pci/intel/ipu-trace.h create mode 100644 drivers/media/pci/intel/ipu.c create mode 100644 drivers/media/pci/intel/ipu.h create mode 100644 drivers/media/pci/intel/ipu6/Makefile create mode 100644 drivers/media/pci/intel/ipu6/ipu-fw-resources.c create mode 100644 drivers/media/pci/intel/ipu6/ipu-platform-buttress-regs.h create mode 100644 drivers/media/pci/intel/ipu6/ipu-platform-isys-csi2-reg.h create mode 100644 drivers/media/pci/intel/ipu6/ipu-platform-isys.h create mode 100644 drivers/media/pci/intel/ipu6/ipu-platform-psys.h create mode 100644 drivers/media/pci/intel/ipu6/ipu-platform-regs.h create mode 100644 drivers/media/pci/intel/ipu6/ipu-platform-resources.h create mode 100644 drivers/media/pci/intel/ipu6/ipu-platform.h create mode 100644 drivers/media/pci/intel/ipu6/ipu-resources.c create mode 100644 drivers/media/pci/intel/ipu6/ipu6-fw-resources.c create mode 100644 drivers/media/pci/intel/ipu6/ipu6-isys-csi2.c create mode 100644 drivers/media/pci/intel/ipu6/ipu6-isys-csi2.h create mode 100644 drivers/media/pci/intel/ipu6/ipu6-isys-gpc.c create mode 100644 drivers/media/pci/intel/ipu6/ipu6-isys-phy.c create mode 100644 drivers/media/pci/intel/ipu6/ipu6-isys-phy.h create mode 100644 drivers/media/pci/intel/ipu6/ipu6-isys.c create mode 100644 drivers/media/pci/intel/ipu6/ipu6-l-scheduler.c create mode 100644 drivers/media/pci/intel/ipu6/ipu6-platform-resources.h create mode 100644 drivers/media/pci/intel/ipu6/ipu6-ppg.c create mode 100644 drivers/media/pci/intel/ipu6/ipu6-ppg.h create mode 100644 drivers/media/pci/intel/ipu6/ipu6-psys-gpc.c create mode 100644 drivers/media/pci/intel/ipu6/ipu6-psys.c create mode 100644 drivers/media/pci/intel/ipu6/ipu6.c create mode 100644 drivers/media/pci/intel/ipu6/ipu6se-fw-resources.c create mode 100644 drivers/media/pci/intel/ipu6/ipu6se-platform-resources.h create mode 100644 drivers/usb/intel_ulpss/Kconfig create mode 100644 drivers/usb/intel_ulpss/Makefile create mode 100644 drivers/usb/intel_ulpss/diag_stub.c create mode 100644 drivers/usb/intel_ulpss/diag_stub.h create mode 100644 drivers/usb/intel_ulpss/gpio_stub.c create mode 100644 drivers/usb/intel_ulpss/gpio_stub.h create mode 100644 drivers/usb/intel_ulpss/i2c_stub.c create mode 100644 drivers/usb/intel_ulpss/i2c_stub.h create mode 100644 drivers/usb/intel_ulpss/mng_stub.c create mode 100644 drivers/usb/intel_ulpss/mng_stub.h create mode 100644 drivers/usb/intel_ulpss/protocol_intel_ulpss.h create mode 100644 drivers/usb/intel_ulpss/ulpss_bridge.c create mode 100644 drivers/usb/intel_ulpss/ulpss_bridge.h create mode 100644 drivers/usb/intel_ulpss/usb_stub.c create mode 100644 drivers/usb/intel_ulpss/usb_stub.h create mode 100644 include/media/ipu-isys.h create mode 100644 include/uapi/linux/ipu-isys.h create mode 100644 include/uapi/linux/ipu-psys.h