From patchwork Wed Oct 13 06:50:07 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Khalid Elmously X-Patchwork-Id: 1540252 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: bilbo.ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=canonical.com header.i=@canonical.com header.a=rsa-sha256 header.s=20210705 header.b=FYArrA8e; dkim-atps=neutral Authentication-Results: ozlabs.org; spf=none (no SPF record) smtp.mailfrom=lists.ubuntu.com (client-ip=91.189.94.19; helo=huckleberry.canonical.com; envelope-from=kernel-team-bounces@lists.ubuntu.com; receiver=) Received: from huckleberry.canonical.com (huckleberry.canonical.com [91.189.94.19]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by bilbo.ozlabs.org (Postfix) with ESMTPS id 4HTjq323pNz9sR4 for ; Wed, 13 Oct 2021 17:50:55 +1100 (AEDT) Received: from localhost ([127.0.0.1] helo=huckleberry.canonical.com) by huckleberry.canonical.com with esmtp (Exim 4.86_2) (envelope-from ) id 1maY6A-0006R3-Di; Wed, 13 Oct 2021 06:50:46 +0000 Received: from smtp-relay-internal-1.internal ([10.131.114.114] helo=smtp-relay-internal-1.canonical.com) by huckleberry.canonical.com with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.86_2) (envelope-from ) id 1maY5p-00069N-84 for kernel-team@lists.ubuntu.com; Wed, 13 Oct 2021 06:50:25 +0000 Received: from mail-qv1-f69.google.com (mail-qv1-f69.google.com [209.85.219.69]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (2048 bits) server-digest SHA256) (No client certificate requested) by smtp-relay-internal-1.canonical.com (Postfix) with ESMTPS id C447B3FFEE for ; Wed, 13 Oct 2021 06:50:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=canonical.com; s=20210705; t=1634107823; bh=8Vn+vAePBeTHziDoYug1I+87xfaTuNI7RcT41niGqnw=; h=From:To:Cc:Subject:Date:Message-Id:In-Reply-To:References: MIME-Version:Content-Type; b=FYArrA8e7EdXv6wYIa9aic+JIHQlKlZqpCiDz4OnX7s74u8aT3w4B+2WjIuoOJ6IX rGfClomPlaxrV7UYrkKDPNW6FTzwArQDgWDMofS3g/gF6CbR3Ir5lJPNlrAM/TYSGH VbZtMr9s0YA0LGFrtGuKrKZ2bt+tVGnfVJxsWeDrkLxTYUjz08OXCPVZeGiDDLQMVm uFkKNJj7GAEIJEqiUUHdLQNni0LVjm+oLz7pm3KgNtD3Xmf6dCmnggFanY+6wlrWcM RxUA3HzBQHuPorlFg6dVeAB/Sb++P7of1c9ugA0KLiUBTGsFeFvoV+cZt1bv6JTc5y uSIYT+nOSFF8A== Received: by mail-qv1-f69.google.com with SMTP id p9-20020a05621421e900b003830bb235fbso1739497qvj.14 for ; Tue, 12 Oct 2021 23:50:23 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references:mime-version:content-transfer-encoding; bh=8Vn+vAePBeTHziDoYug1I+87xfaTuNI7RcT41niGqnw=; b=h1XqjgqW+Pajj2nv5ge4RUa7CHGib4TQjMSG6CvmcC1SsMgu5JfJ603u3k0X1KIIbv bjb2Ukxtxho9asuUO2L6/sPDEdjLH6k4k2G/dxdIvmsQgB0v1XX8sMHkA4BT6t8sx0eF QbpsdfLMfp1cQyihDSDSWmFr93mo9U5i8qd6ZsBhD23KG2T8cKYfqvACD+V/rwmfnF9X YmwEjloNiM4BdnB3VU2ip5q8rUw4r4gQKUEVkz20SK7vBGgHdDR7MZ2mgH+Uf9jVOZRs rnzORtTnhJ/rXSZ6lVo/ZXgl5P/mQ0mzSVs6KA2pwH1wlcDYNz7oBuwygiou3fa3vWDh rRZg== X-Gm-Message-State: AOAM533OVyAC2g/8ZfjfsOiKN9tnyU1MolwMEdlCh6MQVgldpHM8gtzy A1BTqUJdyKfNI8aQCnLiOwjcsF/M5hWkaDPsXI7xkjodFj+bVSIT1RgyD27Iv13P2ePC+0+S+LH DAFNM3OUdkUdMaDXMQUH/RwUGlCIONngqBQ/MmnwNQw== X-Received: by 2002:a05:622a:1105:: with SMTP id e5mr25236080qty.97.1634107822793; Tue, 12 Oct 2021 23:50:22 -0700 (PDT) X-Google-Smtp-Source: ABdhPJx6KZqbVjMCxctxQmf4C+3kex8qb1r/+httlhNX6/N5bB17l3sFfjivJnBmnJ7TCPu7x6y/8Q== X-Received: by 2002:a05:622a:1105:: with SMTP id e5mr25236056qty.97.1634107822519; Tue, 12 Oct 2021 23:50:22 -0700 (PDT) Received: from kbuntu2.fuzzbuzz.org (dhcp-24-53-240-12.cable.user.start.ca. [24.53.240.12]) by smtp.gmail.com with ESMTPSA id t19sm5727298qtn.26.2021.10.12.23.50.21 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Tue, 12 Oct 2021 23:50:22 -0700 (PDT) From: Khalid Elmously To: kernel-team@lists.ubuntu.com Subject: [PATCH 13/13] swiotlb: manipulate orig_addr when tlb_addr has offset Date: Wed, 13 Oct 2021 02:50:07 -0400 Message-Id: <20211013065007.1302-14-khalid.elmously@canonical.com> X-Mailer: git-send-email 2.17.1 In-Reply-To: <20211013065007.1302-1-khalid.elmously@canonical.com> References: <20211013065007.1302-1-khalid.elmously@canonical.com> MIME-Version: 1.0 X-BeenThere: kernel-team@lists.ubuntu.com X-Mailman-Version: 2.1.20 Precedence: list List-Id: Kernel team discussions List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: kernel-team-bounces@lists.ubuntu.com Sender: "kernel-team" From: Bumyong Lee BugLink: https://bugs.launchpad.net/bugs/1943902 in case of driver wants to sync part of ranges with offset, swiotlb_tbl_sync_single() copies from orig_addr base to tlb_addr with offset and ends up with data mismatch. It was removed from "swiotlb: don't modify orig_addr in swiotlb_tbl_sync_single", but said logic has to be added back in. From Linus's email: "That commit which the removed the offset calculation entirely, because the old (unsigned long)tlb_addr & (IO_TLB_SIZE - 1) was wrong, but instead of removing it, I think it should have just fixed it to be (tlb_addr - mem->start) & (IO_TLB_SIZE - 1); instead. That way the slot offset always matches the slot index calculation." (Unfortunatly that broke NVMe). The use-case that drivers are hitting is as follow: 1. Get dma_addr_t from dma_map_single() dma_addr_t tlb_addr = dma_map_single(dev, vaddr, vsize, DMA_TO_DEVICE); |<---------------vsize------------->| +-----------------------------------+ | | original buffer +-----------------------------------+ vaddr swiotlb_align_offset |<----->|<---------------vsize------------->| +-------+-----------------------------------+ | | | swiotlb buffer +-------+-----------------------------------+ tlb_addr 2. Do something 3. Sync dma_addr_t through dma_sync_single_for_device(..) dma_sync_single_for_device(dev, tlb_addr + offset, size, DMA_TO_DEVICE); Error case. Copy data to original buffer but it is from base addr (instead of base addr + offset) in original buffer: swiotlb_align_offset |<----->|<- offset ->|<- size ->| +-------+-----------------------------------+ | | |##########| | swiotlb buffer +-------+-----------------------------------+ tlb_addr |<- size ->| +-----------------------------------+ |##########| | original buffer +-----------------------------------+ vaddr The fix is to copy the data to the original buffer and take into account the offset, like so: swiotlb_align_offset |<----->|<- offset ->|<- size ->| +-------+-----------------------------------+ | | |##########| | swiotlb buffer +-------+-----------------------------------+ tlb_addr |<- offset ->|<- size ->| +-----------------------------------+ | |##########| | original buffer +-----------------------------------+ vaddr [One fix which was Linus's that made more sense to as it created a symmetry would break NVMe. The reason for that is the: unsigned int offset = (tlb_addr - mem->start) & (IO_TLB_SIZE - 1); would come up with the proper offset, but it would lose the alignment (which this patch contains).] Fixes: 16fc3cef33a0 ("swiotlb: don't modify orig_addr in swiotlb_tbl_sync_single") Signed-off-by: Bumyong Lee Signed-off-by: Chanho Park Reviewed-by: Christoph Hellwig Reported-by: Dominique MARTINET Reported-by: Horia Geantă Tested-by: Horia Geantă CC: stable@vger.kernel.org Signed-off-by: Konrad Rzeszutek Wilk (backported from commit 5f89468e2f060031cd89fd4287298e0eaf246bf6) [ kmously: omitted part about alloc_size - not applicable and not needed ] Signed-off-by: Khalid Elmously --- kernel/dma/swiotlb.c | 22 ++++++++++++++-------- 1 file changed, 14 insertions(+), 8 deletions(-) diff --git a/kernel/dma/swiotlb.c b/kernel/dma/swiotlb.c index d8c259c722db5..38971dd61acc7 100644 --- a/kernel/dma/swiotlb.c +++ b/kernel/dma/swiotlb.c @@ -413,6 +413,14 @@ void __init swiotlb_exit(void) swiotlb_cleanup(); } +/* + * Return the offset into a iotlb slot required to keep the device happy. + */ +static unsigned int swiotlb_align_offset(struct device *dev, u64 addr) +{ + return addr & dma_get_min_align_mask(dev) & (IO_TLB_SIZE - 1); +} + /* * Bounce: copy the swiotlb buffer from or back to the original dma location */ @@ -423,10 +431,16 @@ static void swiotlb_bounce(struct device *dev, phys_addr_t tlb_addr, size_t size phys_addr_t orig_addr = io_tlb_orig_addr[index]; unsigned long pfn = PFN_DOWN(orig_addr); unsigned char *vaddr = phys_to_virt(tlb_addr); + unsigned int tlb_offset; if (orig_addr == INVALID_PHYS_ADDR) return; + tlb_offset = (tlb_addr & (IO_TLB_SIZE - 1)) - + swiotlb_align_offset(dev, orig_addr); + + orig_addr += tlb_offset; + if (PageHighMem(pfn_to_page(pfn))) { /* The buffer does not have a mapping. Map it in and copy */ unsigned int offset = orig_addr & ~PAGE_MASK; @@ -460,14 +474,6 @@ static void swiotlb_bounce(struct device *dev, phys_addr_t tlb_addr, size_t size #define slot_addr(start, idx) ((start) + ((idx) << IO_TLB_SHIFT)) -/* - * Return the offset into a iotlb slot required to keep the device happy. - */ -static unsigned int swiotlb_align_offset(struct device *dev, u64 addr) -{ - return addr & dma_get_min_align_mask(dev) & (IO_TLB_SIZE - 1); -} - /* * Carefully handle integer overflow which can occur when boundary_mask == ~0UL. */