From patchwork Sat May 4 08:44:19 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Amjad Alsharafi X-Patchwork-Id: 1931256 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20230601 header.b=DB1DA1t5; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4VWh8N4nxWz1xnS for ; Sat, 4 May 2024 18:45:40 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1s3B0j-00084f-6Z; Sat, 04 May 2024 04:44:49 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1s3B0h-00083k-BL; Sat, 04 May 2024 04:44:47 -0400 Received: from mail-pf1-x433.google.com ([2607:f8b0:4864:20::433]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1s3B0e-0001Vk-W6; Sat, 04 May 2024 04:44:47 -0400 Received: by mail-pf1-x433.google.com with SMTP id d2e1a72fcca58-6edc61d0ff6so410576b3a.2; Sat, 04 May 2024 01:44:44 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1714812282; x=1715417082; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=iFVIMyRlD8/BSmSylP4xvcbNRqi2JvuqJ8ggirzVAcY=; b=DB1DA1t5tXxXOwO/ptWZiy947nvb6Morxt1u/ufhi8PQf6RxSyuYuHN4dOe2/8rEMU 8w/fnB34jChig8d3qzrEpgrwbvxYOKPhI5yOWdxKzaTFG1THzQwl9SowOo+3mepyvb8D RozuItXOmUVXB37WXXdKYG7Yc5BU5i5b5ypg5EQ3Bd9GyymWxuNvAot7NaDhl1IeHUxZ jQhyQ3wy8uaB7vd64ikSm/4Sxn17I4I8JAKk1yVRfdM9TagkVfpz8DsHBLTKwIpS1Wvz dQoedrrqF9UMqtGfXMCmNqXi5Ki+iWPHHGI8sR8GfKeUnN3N8oGYsZG0wMjL34f//wgZ JNgA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1714812282; x=1715417082; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=iFVIMyRlD8/BSmSylP4xvcbNRqi2JvuqJ8ggirzVAcY=; b=SHJBD5/SkIJi5foLC0zAEJSIA6dnhT0PjMq+kIKbA3VCI97HSN08Oq8l/DRrtHqmpH wZi/NL85ap00IypVMH27+7DDUJd7wqVz3y8N8Pcx6NELhBxHuCv6aEKGihfbdiAoXdlX KGLq2Twm2crNfVxi19z9nGbOZt+4Qu+CQvx7pTgSxOeDr81q8MvIaKi3vHFeiVYBxt/T jy1pBCL6VPKAY1FJOJL+m3gSXzybYI66QPFZw5dKcxsiql0cLyhUyZ10PnU9LSGyoszt TlFC1kfQa7vKcahWZ9uLcTewJXnlVt/JSuOcs4OJRs87BpliA2T/y4BnJuuSm/DryVyg Y6Rg== X-Forwarded-Encrypted: i=1; AJvYcCVcVbo/V6bSV6b+yh0bn7O6hXS1V8D4uFeMAYwdTrJ36QLp6aIa32M1kbCF57g6BiKyO8eZiaTPwqMB5MQY9HrGvb5gyfo= X-Gm-Message-State: AOJu0YwCH17n8P1QMYhnVMkWbYXGT/b53HvSHg9IEcBnVeReog7DTKGR FtuRJzQ3Wm4F/5DCAs2pHZkaBb6GhqEQ7lar03WDWZhvDuGSpNTRb8yQc5Jw X-Google-Smtp-Source: AGHT+IGwePShG1wgYeY6sScrGEhr0oOuh+dbaY7QTZCOS0QN0zXsaziZTW5R3tYEVuAky25k8tvIDg== X-Received: by 2002:a05:6a00:10d2:b0:6ea:e2d8:468 with SMTP id d18-20020a056a0010d200b006eae2d80468mr6393129pfu.26.1714812281926; Sat, 04 May 2024 01:44:41 -0700 (PDT) Received: from amjad-pc.. ([192.228.203.229]) by smtp.gmail.com with ESMTPSA id ei32-20020a056a0080e000b006f44e64dfe3sm2310393pfb.177.2024.05.04.01.44.40 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 04 May 2024 01:44:41 -0700 (PDT) From: Amjad Alsharafi To: qemu-devel@nongnu.org Cc: Hanna Reitz , Kevin Wolf , "open list:vvfat" , Amjad Alsharafi Subject: [PATCH v2 1/5] vvfat: Fix bug in writing to middle of file Date: Sat, 4 May 2024 16:44:19 +0800 Message-ID: <2871281d8ea41fb4d7ef8f9beeaba017a1717019.1714811679.git.amjadsharafi10@gmail.com> X-Mailer: git-send-email 2.44.0 In-Reply-To: References: MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::433; envelope-from=amjadsharafi10@gmail.com; helo=mail-pf1-x433.google.com X-Spam_score_int: -17 X-Spam_score: -1.8 X-Spam_bar: - X-Spam_report: (-1.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Before this commit, the behavior when calling `commit_one_file` for example with `offset=0x2000` (second cluster), what will happen is that we won't fetch the next cluster from the fat, and instead use the first cluster for the read operation. This is due to off-by-one error here, where `i=0x2000 !< offset=0x2000`, thus not fetching the next cluster. Signed-off-by: Amjad Alsharafi --- block/vvfat.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/block/vvfat.c b/block/vvfat.c index 9d050ba3ae..ab342f0743 100644 --- a/block/vvfat.c +++ b/block/vvfat.c @@ -2525,7 +2525,7 @@ commit_one_file(BDRVVVFATState* s, int dir_index, uint32_t offset) return -1; } - for (i = s->cluster_size; i < offset; i += s->cluster_size) + for (i = s->cluster_size; i <= offset; i += s->cluster_size) c = modified_fat_get(s, c); fd = qemu_open_old(mapping->path, O_RDWR | O_CREAT | O_BINARY, 0666); From patchwork Sat May 4 08:44:20 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Amjad Alsharafi X-Patchwork-Id: 1931257 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20230601 header.b=Y2Q5xP+N; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4VWh8N75QYz23tp for ; Sat, 4 May 2024 18:45:40 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1s3B0m-00085R-3v; Sat, 04 May 2024 04:44:52 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1s3B0k-00085A-MP; Sat, 04 May 2024 04:44:50 -0400 Received: from mail-oi1-x22d.google.com ([2607:f8b0:4864:20::22d]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1s3B0j-0001WY-7B; Sat, 04 May 2024 04:44:50 -0400 Received: by mail-oi1-x22d.google.com with SMTP id 5614622812f47-3c963880aecso24109b6e.3; Sat, 04 May 2024 01:44:48 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1714812287; x=1715417087; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=ipqqr2xOwh8qICJewRkxYEwaPQGJHL+EZFPMDeo3faU=; b=Y2Q5xP+Nt9YhGame4colnPi5rehUh9jmJR6HzoAp+fv1pwYmGx/W8sIffkb1+m1uoB jWqU5uo2cl4jLidXiimOV78/wZ43+gB7t+eJh/V8ODT5wHlOk9DDl5S36v6apvccxTsD V7D60QOggyCO2TOy6JU0ytwNvoM1wFjUTy967JHI0YMkG486xu6sxKsk4boGHb37pLYw Z5HNWLSb//OiMBTPEGhhMZR2xRnaN2Iyp1BUpTyqRhtTVSGqsRU27mWnFHwfXc784pVQ uVlwWavzEPSb+kALTTsxUktvO0s7l69QDTNjRmo0ZWcPQjP7+v9Ub2iOfrg9iERqYqax /XJg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1714812287; x=1715417087; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=ipqqr2xOwh8qICJewRkxYEwaPQGJHL+EZFPMDeo3faU=; b=mM5lW2yPj9lB09Ocre4LwRZmRdUbq88CnJd5M6hMrD8oNsf+RHScBCwyAr71KvCDvA 8/+hGOozj9vHNW8YrLonaQ/3CfNI0OrP4gw/p9Vdehs7TxvlRhk5NPrMr95re49uGli3 xHxPV/q/a+sLrKE0/FjTT5YldSIQz2ZhFqjjdoi78bnQArDXYq7KoSIJhxhz4JcgGHqx yEBeV+kzIg7LMBmwa13pBWVBaewUSwYT+tJ5UyfC5Jg4plvJfZbVy/SV0yMOrNmdPZws PIt+eJDlJmZmDUGn0ZLggWVbfxv6xEWqwhh4OrWaK4MLAeTJIoDRlJFZ5mg+UaM3IYoH ZJAg== X-Forwarded-Encrypted: i=1; AJvYcCWM3JejuSvsG+TqtPWigva3+RyRcocjnyxw/UfYmx5B6QiMGKC6VrpGlh05RVE4p4RnwHaUPUDhp2k/HGGit36e0XgG1c0= X-Gm-Message-State: AOJu0YwDiGYcI/VF4NSZ/rwbJgtt5UVFNNVaZn7rmY4dIhWw9ZiXnu8a T5Hnx6YnD0wf26LqGcpUKCmnelcjmxjkOxOOrHPU1GAAdyzVxTka30lbjA8k X-Google-Smtp-Source: AGHT+IEFaNWwIjy8dtfZ6D0yRw3I/RI3WdeYhRUMe1JnS8p9jZmN17lbx6FOP4y3ZOFjTxooxyWPJg== X-Received: by 2002:a05:6870:82ac:b0:22e:d541:7eb5 with SMTP id q44-20020a05687082ac00b0022ed5417eb5mr5984365oae.38.1714812287046; Sat, 04 May 2024 01:44:47 -0700 (PDT) Received: from amjad-pc.. ([192.228.203.229]) by smtp.gmail.com with ESMTPSA id ei32-20020a056a0080e000b006f44e64dfe3sm2310393pfb.177.2024.05.04.01.44.45 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 04 May 2024 01:44:46 -0700 (PDT) From: Amjad Alsharafi To: qemu-devel@nongnu.org Cc: Hanna Reitz , Kevin Wolf , "open list:vvfat" , Amjad Alsharafi Subject: [PATCH v2 2/5] vvfat: Fix usage of `info.file.offset` Date: Sat, 4 May 2024 16:44:20 +0800 Message-ID: <836a0ea26f641a4f0ff4b2bd8e9c9d17c59ef2d7.1714811679.git.amjadsharafi10@gmail.com> X-Mailer: git-send-email 2.44.0 In-Reply-To: References: MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::22d; envelope-from=amjadsharafi10@gmail.com; helo=mail-oi1-x22d.google.com X-Spam_score_int: -17 X-Spam_score: -1.8 X-Spam_bar: - X-Spam_report: (-1.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org The field is marked as "the offset in the file (in clusters)", but it was being used like this `cluster_size*(nums)+mapping->info.file.offset`, which is incorrect. Additionally, removed the `abort` when `first_mapping_index` does not match, as this matches the case when adding new clusters for files, and its inevitable that we reach this condition when doing that if the clusters are not after one another, so there is no reason to `abort` here, execution continues and the new clusters are written to disk correctly. Signed-off-by: Amjad Alsharafi --- block/vvfat.c | 9 ++++----- 1 file changed, 4 insertions(+), 5 deletions(-) diff --git a/block/vvfat.c b/block/vvfat.c index ab342f0743..cb3ab81e29 100644 --- a/block/vvfat.c +++ b/block/vvfat.c @@ -1408,7 +1408,7 @@ read_cluster_directory: assert(s->current_fd); - offset=s->cluster_size*(cluster_num-s->current_mapping->begin)+s->current_mapping->info.file.offset; + offset=s->cluster_size*((cluster_num - s->current_mapping->begin) + s->current_mapping->info.file.offset); if(lseek(s->current_fd, offset, SEEK_SET)!=offset) return -3; s->cluster=s->cluster_buffer; @@ -1929,8 +1929,8 @@ get_cluster_count_for_direntry(BDRVVVFATState* s, direntry_t* direntry, const ch (mapping->mode & MODE_DIRECTORY) == 0) { /* was modified in qcow */ - if (offset != mapping->info.file.offset + s->cluster_size - * (cluster_num - mapping->begin)) { + if (offset != s->cluster_size + * ((cluster_num - mapping->begin) + mapping->info.file.offset)) { /* offset of this cluster in file chain has changed */ abort(); copy_it = 1; @@ -1944,7 +1944,6 @@ get_cluster_count_for_direntry(BDRVVVFATState* s, direntry_t* direntry, const ch if (mapping->first_mapping_index != first_mapping_index && mapping->info.file.offset > 0) { - abort(); copy_it = 1; } @@ -2404,7 +2403,7 @@ static int commit_mappings(BDRVVVFATState* s, (mapping->end - mapping->begin); } else next_mapping->info.file.offset = mapping->info.file.offset + - mapping->end - mapping->begin; + (mapping->end - mapping->begin); mapping = next_mapping; } From patchwork Sat May 4 08:44:21 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Amjad Alsharafi X-Patchwork-Id: 1931261 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20230601 header.b=j33RF0C/; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4VWh8t1LD5z1xnS for ; Sat, 4 May 2024 18:46:06 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1s3B0p-00086t-1m; Sat, 04 May 2024 04:44:55 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1s3B0n-00086A-7p; Sat, 04 May 2024 04:44:53 -0400 Received: from mail-pg1-x531.google.com ([2607:f8b0:4864:20::531]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1s3B0l-0001Wz-P2; Sat, 04 May 2024 04:44:53 -0400 Received: by mail-pg1-x531.google.com with SMTP id 41be03b00d2f7-5d4d15ec7c5so350088a12.1; Sat, 04 May 2024 01:44:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1714812289; x=1715417089; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=dq8wco9LxYm2lwgJnPrcuNFswulKN41BsT8tTWCz16o=; b=j33RF0C/TR1r77mNdStr29/A+/wR8bn5iviuuB3aU+mdgTNYaZ0K+TA5YL2EtQsR3w aBn3w4KCKO1wq9SVru5tmn52/PtHywxKiCwFE01YALj/GNQDu6XP2OjS2vnuStE7IP76 Yib+muFIDvP7PGiHQtLz6pQy+qzpwwmMcnvkdR3ioFPOL0y4SsAk+Cjy3reNzI/DVDpy db21+y2zW7SROZEnaK76DXa5Yd7jvdYQejLbGylr+YrCig1G+1ch6PhxVkykesAq8Bg1 tCJJlMVGNYuxJgA8itlgJ3zT+fW8KREp3AiPKNBKri++ZJVEhbTt6dMyvnoQAVNnxqGZ dyYA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1714812289; x=1715417089; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=dq8wco9LxYm2lwgJnPrcuNFswulKN41BsT8tTWCz16o=; b=Zbf00udyo8FnzpU7DowQc0ba+gwSnXavUohbsmIfZQ3zgCdtCH74ii+mOAh1ru8HaL 9LpRlFwkzuMOtvISbntxQ138AMgUpFb9XAzVpXQ9uQYva5gMabW4g/XImAZUXh/o9kmm 4B2bvkhsYmwr7c20d7uNA4k3ql8JJ7L74XD+cd6L6YU1ImmOpIaM/U0mVW2OZgrzKpOe dDK/eXaPVZLq6gYn1XA0EjX9VaefLlA6sL7sxaQieQFrcA3Euec5D9giQqQY5bifwUof DGt0/nAM/FtyzsNHDSJObNeJ/W6Oi7Ye+VoN93gtbHHiEtlxmGS3St8f85m9ITAit/n7 E1ZA== X-Forwarded-Encrypted: i=1; AJvYcCXjQsKMDx7S/B2hwJ2NlLa2oRPMsRxcWCJaAskuG6PlH+JCuszrcGaC+LjO2RqijTk0ZjLeKC/vG0qx6xfdm6nwZd8JVcU= X-Gm-Message-State: AOJu0YzbH6/ZDbZryIygJ0H8zYAG04nbY7SYvaYFQ38oUZME0MWkaG4l UOh0dHe79X0S2p3JQ4OxC4VeQ8y28C05zipVfZFSvuKARl1EO9D4ecsM1oDF X-Google-Smtp-Source: AGHT+IH4P9m+nu650WGmAf809mya2WaEIwjaGyHC6S0sw2KkntiD0A0qJLdsK0rraHauDcEBKTouKw== X-Received: by 2002:a05:6a20:3d85:b0:1aa:755f:1746 with SMTP id s5-20020a056a203d8500b001aa755f1746mr6864350pzi.22.1714812289167; Sat, 04 May 2024 01:44:49 -0700 (PDT) Received: from amjad-pc.. ([192.228.203.229]) by smtp.gmail.com with ESMTPSA id ei32-20020a056a0080e000b006f44e64dfe3sm2310393pfb.177.2024.05.04.01.44.47 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 04 May 2024 01:44:48 -0700 (PDT) From: Amjad Alsharafi To: qemu-devel@nongnu.org Cc: Hanna Reitz , Kevin Wolf , "open list:vvfat" , Amjad Alsharafi Subject: [PATCH v2 3/5] vvfat: Fix reading files with non-continuous clusters Date: Sat, 4 May 2024 16:44:21 +0800 Message-ID: X-Mailer: git-send-email 2.44.0 In-Reply-To: References: MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::531; envelope-from=amjadsharafi10@gmail.com; helo=mail-pg1-x531.google.com X-Spam_score_int: -17 X-Spam_score: -1.8 X-Spam_bar: - X-Spam_report: (-1.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org When reading with `read_cluster` we get the `mapping` with `find_mapping_for_cluster` and then we call `open_file` for this mapping. The issue appear when its the same file, but a second cluster that is not immediately after it, imagine clusters `500 -> 503`, this will give us 2 mappings one has the range `500..501` and another `503..504`, both point to the same file, but different offsets. When we don't open the file since the path is the same, we won't assign `s->current_mapping` and thus accessing way out of bound of the file. From our example above, after `open_file` (that didn't open anything) we will get the offset into the file with `s->cluster_size*(cluster_num-s->current_mapping->begin)`, which will give us `0x2000 * (504-500)`, which is out of bound for this mapping and will produce some issues. Signed-off-by: Amjad Alsharafi --- block/vvfat.c | 21 ++++++++++++++------- 1 file changed, 14 insertions(+), 7 deletions(-) diff --git a/block/vvfat.c b/block/vvfat.c index cb3ab81e29..87165abc26 100644 --- a/block/vvfat.c +++ b/block/vvfat.c @@ -1360,15 +1360,22 @@ static int open_file(BDRVVVFATState* s,mapping_t* mapping) { if(!mapping) return -1; + int new_path = 1; if(!s->current_mapping || - strcmp(s->current_mapping->path,mapping->path)) { - /* open file */ - int fd = qemu_open_old(mapping->path, + s->current_mapping->first_mapping_index!=mapping->first_mapping_index || + (new_path = strcmp(s->current_mapping->path,mapping->path))) { + + if (new_path) { + /* open file */ + int fd = qemu_open_old(mapping->path, O_RDONLY | O_BINARY | O_LARGEFILE); - if(fd<0) - return -1; - vvfat_close_current_file(s); - s->current_fd = fd; + if(fd<0) + return -1; + vvfat_close_current_file(s); + + s->current_fd = fd; + } + assert(s->current_fd); s->current_mapping = mapping; } return 0; From patchwork Sat May 4 08:44:22 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Amjad Alsharafi X-Patchwork-Id: 1931259 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20230601 header.b=i2zonFXm; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4VWh8V1HFWz1xnS for ; Sat, 4 May 2024 18:45:46 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1s3B0t-00088o-AZ; Sat, 04 May 2024 04:44:59 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1s3B0r-00087t-Ge; Sat, 04 May 2024 04:44:57 -0400 Received: from mail-oi1-x235.google.com ([2607:f8b0:4864:20::235]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1s3B0n-0001X8-TG; Sat, 04 May 2024 04:44:57 -0400 Received: by mail-oi1-x235.google.com with SMTP id 5614622812f47-3c86f066256so231046b6e.2; Sat, 04 May 2024 01:44:53 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1714812292; x=1715417092; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=s/zHnNhPumisGq+lPnIFjv3jsyvTJX0d2DVpq+TqD5s=; b=i2zonFXm1SAUFWsAu2vtmlLWSkZ2uDSErbAgGCq2SFSDlobxQnKfaB2ICG9iJkmtho QG+fwXyKJIaFPzgMWe7QXYjP0XezNOXuvi8PfNHu4YlG6kGZemJ+vaD3aHCT0X4x3Q5A +P0NCcOF/xZCov76xP1vw0+A49C3QWNBRwZL/YH9cSKmYKbzgeQdmPTJONqjXmCPQd08 rIDhtfFsrYbg8NydnP+cVebbQywmydAnbVVXiaXELYcugqdDkH0GDl/YGo5UJK5LwJzg jkZfpuTXVDFOwVFXsQDVWsZdb3ZHwk+56pMDQ1lLY1ATfaXEG6Il6dfg9kC3ScJuLIly m5dg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1714812292; x=1715417092; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=s/zHnNhPumisGq+lPnIFjv3jsyvTJX0d2DVpq+TqD5s=; b=vWc6RDoABcDT7uPpcGgMEZPGdHOEm27uLw1Oq0ZKtsi5iG9MInE0iKS2pjPwrgu1lH YpfrZcEY73YZSXB9vdaUZ8gfT/SbuoVH26e4vHMROGNVUwMjPwmju+gvZD99f5KhCd2A h63b8biRo2j7OZD0BQb0VHt+WLWaEB2pHscX98VAkCEFKFa/eY3lQQg5Q0GRUC9m99J4 8jb0q7cxn9v0Se6dKbfRqipktyHeA2ShyLTsXDwd/DaHZNNvD7uHDzhbhudT+a+Dd8d6 G+0e+NZrWRXgQQZGRlb3dfcnYTxO5M2p0znxrw+ubjJCCWa72ux7qPoA+cZmRyvENf// sPTQ== X-Forwarded-Encrypted: i=1; AJvYcCVMZlep940oyIOV/6WhIOa1JYYGVkKm46evT27H3daJPJPAYDpmKErQKaYzW0gAa1TMPA8zcUUNT5xdtekW6qOBNY/d4Uw= X-Gm-Message-State: AOJu0YwdFsmr/E6oCKEmXQuGqkfqasnZJRo1dlFBgcYqFORvBkEF8jn+ Ie8m3DOXYEyx0HteAwIHCUOC25oeUyFmqIKn/RVOh7wVBNVLbVoayXU83bYp X-Google-Smtp-Source: AGHT+IF983eiZi0PtqCgzbcLxuk9TrKzNpAS+927qeMnTDxFnjXTJ/3bsVj94TUq3KzWDfH983DmLQ== X-Received: by 2002:a05:6808:289:b0:3c8:4fc8:1302 with SMTP id z9-20020a056808028900b003c84fc81302mr4864170oic.57.1714812291308; Sat, 04 May 2024 01:44:51 -0700 (PDT) Received: from amjad-pc.. ([192.228.203.229]) by smtp.gmail.com with ESMTPSA id ei32-20020a056a0080e000b006f44e64dfe3sm2310393pfb.177.2024.05.04.01.44.49 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 04 May 2024 01:44:50 -0700 (PDT) From: Amjad Alsharafi To: qemu-devel@nongnu.org Cc: Hanna Reitz , Kevin Wolf , "open list:vvfat" , Amjad Alsharafi Subject: [PATCH v2 4/5] iotests: Add `vvfat` tests Date: Sat, 4 May 2024 16:44:22 +0800 Message-ID: X-Mailer: git-send-email 2.44.0 In-Reply-To: References: MIME-Version: 1.0 Received-SPF: pass client-ip=2607:f8b0:4864:20::235; envelope-from=amjadsharafi10@gmail.com; helo=mail-oi1-x235.google.com X-Spam_score_int: -17 X-Spam_score: -1.8 X-Spam_bar: - X-Spam_report: (-1.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Added several tests to verify the implementation of the vvfat driver. We needed a way to interact with it, so created a basic `fat16.py` driver that handled writing correct sectors for us. Signed-off-by: Amjad Alsharafi --- tests/qemu-iotests/check | 2 +- tests/qemu-iotests/fat16.py | 507 +++++++++++++++++++++++++++++ tests/qemu-iotests/tests/vvfat | 400 +++++++++++++++++++++++ tests/qemu-iotests/tests/vvfat.out | 5 + 4 files changed, 913 insertions(+), 1 deletion(-) create mode 100644 tests/qemu-iotests/fat16.py create mode 100755 tests/qemu-iotests/tests/vvfat create mode 100755 tests/qemu-iotests/tests/vvfat.out diff --git a/tests/qemu-iotests/check b/tests/qemu-iotests/check index 56d88ca423..545f9ec7bd 100755 --- a/tests/qemu-iotests/check +++ b/tests/qemu-iotests/check @@ -84,7 +84,7 @@ def make_argparser() -> argparse.ArgumentParser: p.set_defaults(imgfmt='raw', imgproto='file') format_list = ['raw', 'bochs', 'cloop', 'parallels', 'qcow', 'qcow2', - 'qed', 'vdi', 'vpc', 'vhdx', 'vmdk', 'luks', 'dmg'] + 'qed', 'vdi', 'vpc', 'vhdx', 'vmdk', 'luks', 'dmg', 'vvfat'] g_fmt = p.add_argument_group( ' image format options', 'The following options set the IMGFMT environment variable. ' diff --git a/tests/qemu-iotests/fat16.py b/tests/qemu-iotests/fat16.py new file mode 100644 index 0000000000..6ac5508d8d --- /dev/null +++ b/tests/qemu-iotests/fat16.py @@ -0,0 +1,507 @@ +# A simple FAT16 driver that is used to test the `vvfat` driver in QEMU. +# +# Copyright (C) 2024 Amjad Alsharafi +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . + +from typing import List + +SECTOR_SIZE = 512 +DIRENTRY_SIZE = 32 + + +class MBR: + def __init__(self, data: bytes): + assert len(data) == 512 + self.partition_table = [] + for i in range(4): + partition = data[446 + i * 16 : 446 + (i + 1) * 16] + self.partition_table.append( + { + "status": partition[0], + "start_head": partition[1], + "start_sector": partition[2] & 0x3F, + "start_cylinder": ((partition[2] & 0xC0) << 2) | partition[3], + "type": partition[4], + "end_head": partition[5], + "end_sector": partition[6] & 0x3F, + "end_cylinder": ((partition[6] & 0xC0) << 2) | partition[7], + "start_lba": int.from_bytes(partition[8:12], "little"), + "size": int.from_bytes(partition[12:16], "little"), + } + ) + + def __str__(self): + return "\n".join( + [f"{i}: {partition}" for i, partition in enumerate(self.partition_table)] + ) + + +class FatBootSector: + def __init__(self, data: bytes): + assert len(data) == 512 + self.bytes_per_sector = int.from_bytes(data[11:13], "little") + self.sectors_per_cluster = data[13] + self.reserved_sectors = int.from_bytes(data[14:16], "little") + self.fat_count = data[16] + self.root_entries = int.from_bytes(data[17:19], "little") + self.media_descriptor = data[21] + self.fat_size = int.from_bytes(data[22:24], "little") + self.sectors_per_fat = int.from_bytes(data[22:24], "little") + self.sectors_per_track = int.from_bytes(data[24:26], "little") + self.heads = int.from_bytes(data[26:28], "little") + self.hidden_sectors = int.from_bytes(data[28:32], "little") + self.total_sectors = int.from_bytes(data[32:36], "little") + self.drive_number = data[36] + self.volume_id = int.from_bytes(data[39:43], "little") + self.volume_label = data[43:54].decode("ascii").strip() + self.fs_type = data[54:62].decode("ascii").strip() + + def root_dir_start(self): + """ + Calculate the start sector of the root directory. + """ + return self.reserved_sectors + self.fat_count * self.sectors_per_fat + + def root_dir_size(self): + """ + Calculate the size of the root directory in sectors. + """ + return ( + self.root_entries * DIRENTRY_SIZE + self.bytes_per_sector - 1 + ) // self.bytes_per_sector + + def data_sector_start(self): + """ + Calculate the start sector of the data region. + """ + return self.root_dir_start() + self.root_dir_size() + + def first_sector_of_cluster(self, cluster: int): + """ + Calculate the first sector of the given cluster. + """ + return self.data_sector_start() + (cluster - 2) * self.sectors_per_cluster + + def cluster_bytes(self): + """ + Calculate the number of bytes in a cluster. + """ + return self.bytes_per_sector * self.sectors_per_cluster + + def __str__(self): + return ( + f"Bytes per sector: {self.bytes_per_sector}\n" + f"Sectors per cluster: {self.sectors_per_cluster}\n" + f"Reserved sectors: {self.reserved_sectors}\n" + f"FAT count: {self.fat_count}\n" + f"Root entries: {self.root_entries}\n" + f"Total sectors: {self.total_sectors}\n" + f"Media descriptor: {self.media_descriptor}\n" + f"Sectors per FAT: {self.sectors_per_fat}\n" + f"Sectors per track: {self.sectors_per_track}\n" + f"Heads: {self.heads}\n" + f"Hidden sectors: {self.hidden_sectors}\n" + f"Drive number: {self.drive_number}\n" + f"Volume ID: {self.volume_id}\n" + f"Volume label: {self.volume_label}\n" + f"FS type: {self.fs_type}\n" + ) + + +class FatDirectoryEntry: + def __init__(self, data: bytes, sector: int, offset: int): + self.name = data[0:8].decode("ascii").strip() + self.ext = data[8:11].decode("ascii").strip() + self.attributes = data[11] + self.reserved = data[12] + self.create_time_tenth = data[13] + self.create_time = int.from_bytes(data[14:16], "little") + self.create_date = int.from_bytes(data[16:18], "little") + self.last_access_date = int.from_bytes(data[18:20], "little") + high_cluster = int.from_bytes(data[20:22], "little") + self.last_mod_time = int.from_bytes(data[22:24], "little") + self.last_mod_date = int.from_bytes(data[24:26], "little") + low_cluster = int.from_bytes(data[26:28], "little") + self.cluster = (high_cluster << 16) | low_cluster + self.size_bytes = int.from_bytes(data[28:32], "little") + + # extra (to help write back to disk) + self.sector = sector + self.offset = offset + + def as_bytes(self) -> bytes: + return ( + self.name.ljust(8, " ").encode("ascii") + + self.ext.ljust(3, " ").encode("ascii") + + self.attributes.to_bytes(1, "little") + + self.reserved.to_bytes(1, "little") + + self.create_time_tenth.to_bytes(1, "little") + + self.create_time.to_bytes(2, "little") + + self.create_date.to_bytes(2, "little") + + self.last_access_date.to_bytes(2, "little") + + (self.cluster >> 16).to_bytes(2, "little") + + self.last_mod_time.to_bytes(2, "little") + + self.last_mod_date.to_bytes(2, "little") + + (self.cluster & 0xFFFF).to_bytes(2, "little") + + self.size_bytes.to_bytes(4, "little") + ) + + def whole_name(self): + if self.ext: + return f"{self.name}.{self.ext}" + else: + return self.name + + def __str__(self): + return ( + f"Name: {self.name}\n" + f"Ext: {self.ext}\n" + f"Attributes: {self.attributes}\n" + f"Reserved: {self.reserved}\n" + f"Create time tenth: {self.create_time_tenth}\n" + f"Create time: {self.create_time}\n" + f"Create date: {self.create_date}\n" + f"Last access date: {self.last_access_date}\n" + f"Last mod time: {self.last_mod_time}\n" + f"Last mod date: {self.last_mod_date}\n" + f"Cluster: {self.cluster}\n" + f"Size: {self.size_bytes}\n" + ) + + def __repr__(self): + # convert to dict + return str(vars(self)) + + +class Fat16: + def __init__( + self, + start_sector: int, + size: int, + sector_reader: callable, + sector_writer: callable, + ): + self.start_sector = start_sector + self.size_in_sectors = size + self.sector_reader = sector_reader + self.sector_writer = sector_writer + + self.boot_sector = FatBootSector(self.sector_reader(start_sector)) + + fat_size_in_sectors = self.boot_sector.fat_size * self.boot_sector.fat_count + self.fats = self.read_sectors( + self.boot_sector.reserved_sectors, fat_size_in_sectors + ) + self.fats_dirty_sectors = set() + + def read_sectors(self, start_sector: int, num_sectors: int) -> bytes: + return self.sector_reader(start_sector + self.start_sector, num_sectors) + + def write_sectors(self, start_sector: int, data: bytes): + return self.sector_writer(start_sector + self.start_sector, data) + + def directory_from_bytes( + self, data: bytes, start_sector: int + ) -> List[FatDirectoryEntry]: + """ + Convert `bytes` into a list of `FatDirectoryEntry` objects. + Will ignore long file names. + Will stop when it encounters a 0x00 byte. + """ + + entries = [] + for i in range(0, len(data), DIRENTRY_SIZE): + entry = data[i : i + DIRENTRY_SIZE] + + current_sector = start_sector + (i // SECTOR_SIZE) + current_offset = i % SECTOR_SIZE + + if entry[0] == 0: + break + elif entry[0] == 0xE5: + # Deleted file + continue + + if entry[11] & 0xF == 0xF: + # Long file name + continue + + entries.append(FatDirectoryEntry(entry, current_sector, current_offset)) + return entries + + def read_root_directory(self) -> List[FatDirectoryEntry]: + root_dir = self.read_sectors( + self.boot_sector.root_dir_start(), self.boot_sector.root_dir_size() + ) + return self.directory_from_bytes(root_dir, self.boot_sector.root_dir_start()) + + def read_fat_entry(self, cluster: int) -> int: + """ + Read the FAT entry for the given cluster. + """ + fat_offset = cluster * 2 # FAT16 + return int.from_bytes(self.fats[fat_offset : fat_offset + 2], "little") + + def write_fat_entry(self, cluster: int, value: int): + """ + Write the FAT entry for the given cluster. + """ + fat_offset = cluster * 2 + self.fats = ( + self.fats[:fat_offset] + + value.to_bytes(2, "little") + + self.fats[fat_offset + 2 :] + ) + self.fats_dirty_sectors.add(fat_offset // SECTOR_SIZE) + + def flush_fats(self): + """ + Write the FATs back to the disk. + """ + for sector in self.fats_dirty_sectors: + data = self.fats[sector * SECTOR_SIZE : (sector + 1) * SECTOR_SIZE] + sector = self.boot_sector.reserved_sectors + sector + self.write_sectors(sector, data) + self.fats_dirty_sectors = set() + + def next_cluster(self, cluster: int) -> int | None: + """ + Get the next cluster in the chain. + If its `None`, then its the last cluster. + The function will crash if the next cluster is `FREE` (unexpected) or invalid entry. + """ + fat_entry = self.read_fat_entry(cluster) + if fat_entry == 0: + raise Exception("Unexpected: FREE cluster") + elif fat_entry == 1: + raise Exception("Unexpected: RESERVED cluster") + elif fat_entry >= 0xFFF8: + return None + elif fat_entry >= 0xFFF7: + raise Exception("Invalid FAT entry") + else: + return fat_entry + + def next_free_cluster(self) -> int: + """ + Find the next free cluster. + """ + # simple linear search + for i in range(2, 0xFFFF): + if self.read_fat_entry(i) == 0: + return i + raise Exception("No free clusters") + + def read_cluster(self, cluster: int) -> bytes: + """ + Read the cluster at the given cluster. + """ + return self.read_sectors( + self.boot_sector.first_sector_of_cluster(cluster), + self.boot_sector.sectors_per_cluster, + ) + + def write_cluster(self, cluster: int, data: bytes): + """ + Write the cluster at the given cluster. + """ + assert len(data) == self.boot_sector.cluster_bytes() + return self.write_sectors( + self.boot_sector.first_sector_of_cluster(cluster), + data, + ) + + def read_directory(self, cluster: int) -> List[FatDirectoryEntry]: + """ + Read the directory at the given cluster. + """ + entries = [] + while cluster is not None: + data = self.read_cluster(cluster) + entries.extend( + self.directory_from_bytes( + data, self.boot_sector.first_sector_of_cluster(cluster) + ) + ) + cluster = self.next_cluster(cluster) + return entries + + def update_direntry(self, entry: FatDirectoryEntry): + """ + Write the directory entry back to the disk. + """ + sector = self.read_sectors(entry.sector, 1) + sector = ( + sector[: entry.offset] + + entry.as_bytes() + + sector[entry.offset + DIRENTRY_SIZE :] + ) + self.write_sectors(entry.sector, sector) + + def find_direntry(self, path: str) -> FatDirectoryEntry | None: + """ + Find the directory entry for the given path. + """ + assert path[0] == "/", "Path must start with /" + + path = path[1:] # remove the leading / + parts = path.split("/") + directory = self.read_root_directory() + + current_entry = None + + for i, part in enumerate(parts): + is_last = i == len(parts) - 1 + + for entry in directory: + if entry.whole_name() == part: + current_entry = entry + break + if current_entry is None: + return None + + if is_last: + return current_entry + else: + if current_entry.attributes & 0x10 == 0: + raise Exception(f"{current_entry.whole_name()} is not a directory") + else: + directory = self.read_directory(current_entry.cluster) + + def read_file(self, entry: FatDirectoryEntry) -> bytes: + """ + Read the content of the file at the given path. + """ + if entry is None: + return None + if entry.attributes & 0x10 != 0: + raise Exception(f"{entry.whole_name()} is a directory") + + data = b"" + cluster = entry.cluster + while cluster is not None and len(data) <= entry.size_bytes: + data += self.read_cluster(cluster) + cluster = self.next_cluster(cluster) + return data[: entry.size_bytes] + + def truncate_file(self, entry: FatDirectoryEntry, new_size: int): + """ + Truncate the file at the given path to the new size. + """ + if entry is None: + return Exception("entry is None") + if entry.attributes & 0x10 != 0: + raise Exception(f"{entry.whole_name()} is a directory") + + def clusters_from_size(size: int): + return (size + self.boot_sector.cluster_bytes() - 1) // self.boot_sector.cluster_bytes() + + + # First, allocate new FATs if we need to + required_clusters = clusters_from_size(new_size) + current_clusters = clusters_from_size(entry.size_bytes) + + affected_clusters = set() + + # Keep at least one cluster, easier to manage this way + if required_clusters == 0: + required_clusters = 1 + if current_clusters == 0: + current_clusters = 1 + + if required_clusters > current_clusters: + # Allocate new clusters + cluster = entry.cluster + to_add = required_clusters + for _ in range(current_clusters - 1): + to_add -= 1 + cluster = self.next_cluster(cluster) + assert required_clusters > 0, "No new clusters to allocate" + assert cluster is not None, "Cluster is None" + assert self.next_cluster(cluster) is None, "Cluster is not the last cluster" + + # Allocate new clusters + for _ in range(to_add - 1): + new_cluster = self.next_free_cluster() + self.write_fat_entry(cluster, new_cluster) + self.write_fat_entry(new_cluster, 0xFFFF) + cluster = new_cluster + + elif required_clusters < current_clusters: + # Truncate the file + cluster = entry.cluster + for _ in range(required_clusters - 1): + cluster = self.next_cluster(cluster) + assert cluster is not None, "Cluster is None" + + next_cluster = self.next_cluster(cluster) + # mark last as EOF + self.write_fat_entry(cluster, 0xFFFF) + # free the rest + while next_cluster is not None: + cluster = next_cluster + next_cluster = self.next_cluster(next_cluster) + self.write_fat_entry(cluster, 0) + + self.flush_fats() + + # verify number of clusters + cluster = entry.cluster + count = 0 + while cluster is not None: + count += 1 + affected_clusters.add(cluster) + cluster = self.next_cluster(cluster) + assert count == required_clusters, f"Expected {required_clusters} clusters, got {count}" + + # update the size + entry.size_bytes = new_size + self.update_direntry(entry) + + # trigger every affected cluster + for cluster in affected_clusters: + first_sector = self.boot_sector.first_sector_of_cluster(cluster) + first_sector_data = self.read_sectors(first_sector, 1) + self.write_sectors(first_sector, first_sector_data) + + def write_file(self, entry: FatDirectoryEntry, data: bytes): + """ + Write the content of the file at the given path. + """ + if entry is None: + return Exception("entry is None") + if entry.attributes & 0x10 != 0: + raise Exception(f"{entry.whole_name()} is a directory") + + data_len = len(data) + + self.truncate_file(entry, data_len) + + cluster = entry.cluster + while cluster is not None: + data_to_write = data[: self.boot_sector.cluster_bytes()] + last_data = False + if len(data_to_write) < self.boot_sector.cluster_bytes(): + last_data = True + old_data = self.read_cluster(cluster) + data_to_write += old_data[len(data_to_write) :] + + self.write_cluster(cluster, data_to_write) + data = data[self.boot_sector.cluster_bytes() :] + if len(data) == 0: + break + cluster = self.next_cluster(cluster) + + assert len(data) == 0, "Data was not written completely, clusters missing" diff --git a/tests/qemu-iotests/tests/vvfat b/tests/qemu-iotests/tests/vvfat new file mode 100755 index 0000000000..e0e23d1ab8 --- /dev/null +++ b/tests/qemu-iotests/tests/vvfat @@ -0,0 +1,400 @@ +#!/usr/bin/env python3 +# group: rw vvfat +# +# Test vvfat driver implementation +# Here, we use a simple FAT16 implementation and check the behavior of the vvfat driver. +# +# Copyright (C) 2024 Amjad Alsharafi +# +# This program is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . + +import os, shutil +import iotests +from iotests import imgfmt, QMPTestCase +from fat16 import MBR, Fat16, DIRENTRY_SIZE + +filesystem = os.path.join(iotests.test_dir, "filesystem") + +nbd_sock = iotests.file_path("nbd.sock", base_dir=iotests.sock_dir) +nbd_uri = "nbd+unix:///disk?socket=" + nbd_sock + +SECTOR_SIZE = 512 + + +class TestVVFatDriver(QMPTestCase): + def setUp(self) -> None: + if os.path.exists(filesystem): + if os.path.isdir(filesystem): + shutil.rmtree(filesystem) + else: + print(f"Error: {filesystem} exists and is not a directory") + exit(1) + os.mkdir(filesystem) + + # Add some text files to the filesystem + for i in range(10): + with open(os.path.join(filesystem, f"file{i}.txt"), "w") as f: + f.write(f"Hello, world! {i}\n") + + # Add 2 large files, above the cluster size (8KB) + with open(os.path.join(filesystem, "large1.txt"), "wb") as f: + # write 'A' * 1KB, 'B' * 1KB, 'C' * 1KB, ... + for i in range(8 * 2): # two clusters + f.write(bytes([0x41 + i] * 1024)) + + with open(os.path.join(filesystem, "large2.txt"), "wb") as f: + # write 'A' * 1KB, 'B' * 1KB, 'C' * 1KB, ... + for i in range(8 * 3): # 3 clusters + f.write(bytes([0x41 + i] * 1024)) + + self.vm = iotests.VM() + + self.vm.add_blockdev( + self.vm.qmp_to_opts( + { + "driver": imgfmt, + "node-name": "disk", + "rw": "true", + "fat-type": "16", + "dir": filesystem, + } + ) + ) + + self.vm.launch() + + self.vm.qmp_log("block-dirty-bitmap-add", **{"node": "disk", "name": "bitmap0"}) + + # attach nbd server + self.vm.qmp_log( + "nbd-server-start", + **{"addr": {"type": "unix", "data": {"path": nbd_sock}}}, + filters=[], + ) + + self.vm.qmp_log( + "nbd-server-add", + **{"device": "disk", "writable": True, "bitmap": "bitmap0"}, + ) + + self.qio = iotests.QemuIoInteractive("-f", "raw", nbd_uri) + + def tearDown(self) -> None: + self.qio.close() + self.vm.shutdown() + # print(self.vm.get_log()) + shutil.rmtree(filesystem) + + def read_sectors(self, sector: int, num: int = 1) -> bytes: + """ + Read `num` sectors starting from `sector` from the `disk`. + This uses `QemuIoInteractive` to read the sectors into `stdout` and then parse the output. + """ + self.assertGreater(num, 0) + # The output contains the content of the sector in hex dump format + # We need to extract the content from it + output = self.qio.cmd(f"read -v {sector * SECTOR_SIZE} {num * SECTOR_SIZE}") + # Each row is 16 bytes long, and we are writing `num` sectors + rows = num * SECTOR_SIZE // 16 + output_rows = output.split("\n")[:rows] + + hex_content = "".join( + [(row.split(": ")[1]).split(" ")[0] for row in output_rows] + ) + bytes_content = bytes.fromhex(hex_content) + + self.assertEqual(len(bytes_content), num * SECTOR_SIZE) + + return bytes_content + + def write_sectors(self, sector: int, data: bytes): + """ + Write `data` to the `disk` starting from `sector`. + This uses `QemuIoInteractive` to write the data into the disk. + """ + + self.assertGreater(len(data), 0) + self.assertEqual(len(data) % SECTOR_SIZE, 0) + + temp_file = os.path.join(iotests.test_dir, "temp.bin") + with open(temp_file, "wb") as f: + f.write(data) + + self.qio.cmd(f"write -s {temp_file} {sector * SECTOR_SIZE} {len(data)}") + + os.remove(temp_file) + + def init_fat16(self): + mbr = MBR(self.read_sectors(0)) + return Fat16( + mbr.partition_table[0]["start_lba"], + mbr.partition_table[0]["size"], + self.read_sectors, + self.write_sectors, + ) + + # Tests + + def test_fat_filesystem(self): + """ + Test that vvfat produce a valid FAT16 and MBR sectors + """ + mbr = MBR(self.read_sectors(0)) + + self.assertEqual(mbr.partition_table[0]["status"], 0x80) + self.assertEqual(mbr.partition_table[0]["type"], 6) + + fat16 = Fat16( + mbr.partition_table[0]["start_lba"], + mbr.partition_table[0]["size"], + self.read_sectors, + self.write_sectors, + ) + self.assertEqual(fat16.boot_sector.bytes_per_sector, 512) + self.assertEqual(fat16.boot_sector.volume_label, "QEMU VVFAT") + + def test_read_root_directory(self): + """ + Test the content of the root directory + """ + fat16 = self.init_fat16() + + root_dir = fat16.read_root_directory() + + self.assertEqual(len(root_dir), 13) # 12 + 1 special file + + files = { + "QEMU VVF.AT": 0, # special empty file + "FILE0.TXT": 16, + "FILE1.TXT": 16, + "FILE2.TXT": 16, + "FILE3.TXT": 16, + "FILE4.TXT": 16, + "FILE5.TXT": 16, + "FILE6.TXT": 16, + "FILE7.TXT": 16, + "FILE8.TXT": 16, + "FILE9.TXT": 16, + "LARGE1.TXT": 0x2000 * 2, + "LARGE2.TXT": 0x2000 * 3, + } + + for entry in root_dir: + self.assertIn(entry.whole_name(), files) + self.assertEqual(entry.size_bytes, files[entry.whole_name()]) + + def test_direntry_as_bytes(self): + """ + Test if we can convert Direntry back to bytes, so that we can write it back to the disk safely. + """ + fat16 = self.init_fat16() + + root_dir = fat16.read_root_directory() + first_entry_bytes = fat16.read_sectors(fat16.boot_sector.root_dir_start(), 1) + # The first entry won't be deleted, so we can compare it with the first entry in the root directory + self.assertEqual(root_dir[0].as_bytes(), first_entry_bytes[:DIRENTRY_SIZE]) + + def test_read_files(self): + """ + Test reading the content of the files + """ + fat16 = self.init_fat16() + + for i in range(10): + file = fat16.find_direntry(f"/FILE{i}.TXT") + self.assertIsNotNone(file) + self.assertEqual( + fat16.read_file(file), f"Hello, world! {i}\n".encode("ascii") + ) + + # test large files + large1 = fat16.find_direntry("/LARGE1.TXT") + with open(os.path.join(filesystem, "large1.txt"), "rb") as f: + self.assertEqual(fat16.read_file(large1), f.read()) + + large2 = fat16.find_direntry("/LARGE2.TXT") + self.assertIsNotNone(large2) + with open(os.path.join(filesystem, "large2.txt"), "rb") as f: + self.assertEqual(fat16.read_file(large2), f.read()) + + def test_write_file_same_content_direct(self): + """ + Similar to `test_write_file_in_same_content`, but we write the file directly clusters + and thus we don't go through the modification of direntry. + """ + fat16 = self.init_fat16() + + file = fat16.find_direntry("/FILE0.TXT") + self.assertIsNotNone(file) + + data = fat16.read_cluster(file.cluster) + fat16.write_cluster(file.cluster, data) + + with open(os.path.join(filesystem, "file0.txt"), "rb") as f: + self.assertEqual(fat16.read_file(file), f.read()) + + def test_write_file_in_same_content(self): + """ + Test writing the same content to the file back to it + """ + fat16 = self.init_fat16() + + file = fat16.find_direntry("/FILE0.TXT") + self.assertIsNotNone(file) + + self.assertEqual(fat16.read_file(file), b"Hello, world! 0\n") + + fat16.write_file(file, b"Hello, world! 0\n") + + self.assertEqual(fat16.read_file(file), b"Hello, world! 0\n") + + with open(os.path.join(filesystem, "file0.txt"), "rb") as f: + self.assertEqual(f.read(), b"Hello, world! 0\n") + + def test_modify_content_same_clusters(self): + """ + Test modifying the content of the file without changing the number of clusters + """ + fat16 = self.init_fat16() + + file = fat16.find_direntry("/FILE0.TXT") + self.assertIsNotNone(file) + + new_content = b"Hello, world! Modified\n" + self.assertEqual(fat16.read_file(file), b"Hello, world! 0\n") + + fat16.write_file(file, new_content) + + self.assertEqual(fat16.read_file(file), new_content) + with open(os.path.join(filesystem, "file0.txt"), "rb") as f: + self.assertEqual(f.read(), new_content) + + def test_truncate_file_same_clusters_less(self): + """ + Test truncating the file without changing number of clusters + Test decreasing the file size + """ + fat16 = self.init_fat16() + + file = fat16.find_direntry("/FILE0.TXT") + self.assertIsNotNone(file) + + self.assertEqual(fat16.read_file(file), b"Hello, world! 0\n") + + fat16.truncate_file(file, 5) + + new_content = fat16.read_file(file) + + self.assertEqual(new_content, b"Hello") + + with open(os.path.join(filesystem, "file0.txt"), "rb") as f: + self.assertEqual(f.read(), new_content) + + def test_truncate_file_same_clusters_more(self): + """ + Test truncating the file without changing number of clusters + Test increase the file size + """ + fat16 = self.init_fat16() + + file = fat16.find_direntry("/FILE0.TXT") + self.assertIsNotNone(file) + + self.assertEqual(fat16.read_file(file), b"Hello, world! 0\n") + + fat16.truncate_file(file, 20) + + new_content = fat16.read_file(file) + + # random pattern will be appended to the file, and its not always the same + self.assertEqual(new_content[:16], b"Hello, world! 0\n") + self.assertEqual(len(new_content), 20) + + with open(os.path.join(filesystem, "file0.txt"), "rb") as f: + self.assertEqual(f.read(), new_content) + + def test_write_large_file(self): + """ + Test writing a large file + """ + fat16 = self.init_fat16() + + file = fat16.find_direntry("/LARGE1.TXT") + self.assertIsNotNone(file) + + # The content of LARGE1 is A * 1KB, B * 1KB, C * 1KB, ..., P * 1KB + # Lets change it to be Z * 1KB, Y * 1KB, X * 1KB, ..., K * 1KB + # without changing the number of clusters or filesize + new_content = b"".join([bytes([0x5A - i] * 1024) for i in range(16)]) + + fat16.write_file(file, new_content) + + with open(os.path.join(filesystem, "large1.txt"), "rb") as f: + self.assertEqual(f.read(), new_content) + + def test_truncate_file_change_clusters_less(self): + """ + Test truncating a file by reducing the number of clusters + """ + fat16 = self.init_fat16() + + file = fat16.find_direntry("/LARGE1.TXT") + self.assertIsNotNone(file) + + fat16.truncate_file(file, 1) + + self.assertEqual(fat16.read_file(file), b"A") + + with open(os.path.join(filesystem, "large1.txt"), "rb") as f: + self.assertEqual(f.read(), b"A") + + + def test_write_file_change_clusters_less(self): + """ + Test truncating a file by reducing the number of clusters + """ + fat16 = self.init_fat16() + + file = fat16.find_direntry("/LARGE2.TXT") + self.assertIsNotNone(file) + + new_content = b"Hello, world! This was a large file\n" + new_content = b"Z" * 8 * 1024 * 2 + + fat16.write_file(file, new_content) + + with open(os.path.join(filesystem, "large2.txt"), "rb") as f: + self.assertEqual(f.read(), new_content) + + def test_write_file_change_clusters_more(self): + """ + Test truncating a file by increasing the number of clusters + """ + fat16 = self.init_fat16() + + file = fat16.find_direntry("/LARGE2.TXT") + self.assertIsNotNone(file) + + new_content = b"Z" * 8 * 1024 * 4 + + fat16.write_file(file, new_content) + + with open(os.path.join(filesystem, "large2.txt"), "rb") as f: + self.assertEqual(f.read(), new_content) + + + +if __name__ == "__main__": + # This is a specific test for vvfat driver + iotests.main(supported_fmts=["vvfat"], supported_protocols=["file"]) diff --git a/tests/qemu-iotests/tests/vvfat.out b/tests/qemu-iotests/tests/vvfat.out new file mode 100755 index 0000000000..fa16b5ccef --- /dev/null +++ b/tests/qemu-iotests/tests/vvfat.out @@ -0,0 +1,5 @@ +............. +---------------------------------------------------------------------- +Ran 13 tests + +OK From patchwork Sat May 4 08:44:23 2024 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Amjad Alsharafi X-Patchwork-Id: 1931260 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@legolas.ozlabs.org Authentication-Results: legolas.ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.a=rsa-sha256 header.s=20230601 header.b=TAY09B5M; dkim-atps=neutral Authentication-Results: legolas.ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=nongnu.org (client-ip=209.51.188.17; helo=lists.gnu.org; envelope-from=qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org; receiver=patchwork.ozlabs.org) Received: from lists.gnu.org (lists.gnu.org [209.51.188.17]) (using TLSv1.2 with cipher ECDHE-ECDSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by legolas.ozlabs.org (Postfix) with ESMTPS id 4VWh8l5wLWz1xnS for ; Sat, 4 May 2024 18:45:59 +1000 (AEST) Received: from localhost ([::1] helo=lists1p.gnu.org) by lists.gnu.org with esmtp (Exim 4.90_1) (envelope-from ) id 1s3B0s-00088k-S7; Sat, 04 May 2024 04:44:58 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from ) id 1s3B0r-00088A-Tx; Sat, 04 May 2024 04:44:57 -0400 Received: from mail-oa1-x2b.google.com ([2001:4860:4864:20::2b]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128) (Exim 4.90_1) (envelope-from ) id 1s3B0p-0001XZ-KL; Sat, 04 May 2024 04:44:57 -0400 Received: by mail-oa1-x2b.google.com with SMTP id 586e51a60fabf-23eb1851c34so56378fac.0; Sat, 04 May 2024 01:44:54 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20230601; t=1714812294; x=1715417094; darn=nongnu.org; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:from:to:cc:subject:date :message-id:reply-to; bh=9gTE/AfpQPUSDgNS6a4+2A3kt7wg96uujAYNI02kwlk=; b=TAY09B5M3rjMdLSEVmlxRKQs03z7vEcOTSApSqJv/GY5+5Prp+qbbx8sJJgA4iU3n1 R6aEzaZddM/4munF6zmf2zzvY+1udAjrkiWclAIx+BnidWZn8dua+WMs2t0NZ9jvHayz hjP8CfILo1r1eSfS8NjNVbYnq3RJ5OjF20XTI1YCmGic0A2LNvl0MaNvWWjEsmlHb5Gu 7SL9iUY7LcIfsp3TtLNDxR5BH3fUjHbxnpgTB1t6bxnk2K2MNGDf7txdCOLaVFoKgZop tRFIwn7GKman9FmDKczbPbpsAvmp6WdEuGHod26QDSk3LhyS6c0Nio7yozCEO7vlLtcn Ju1w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1714812294; x=1715417094; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:x-gm-message-state:from:to:cc :subject:date:message-id:reply-to; bh=9gTE/AfpQPUSDgNS6a4+2A3kt7wg96uujAYNI02kwlk=; b=tWrrGEO71BRWshcu707eKi1XiywpwCed21f8iWkPWCOu6MB/Ah5JNI5IHKZUjB1tMJ NfEQkaixOWa+ZND+b+xh2fUQvnH3IpQvZmLut5z50DnsRxdiQSBdAPVUgyU6/37TQoEk b2O+6JmTN3JD3BYAasmi0CKQ520XjtnNbJayzUn1VoCIxZbkllDwTsVLPYkbiWBhjx86 2Stq6KQ2ufapk7sfJvziFicMEp9fBro/+N96zSKjdcyW913FVtLi8Q1uKtNn4I0vEBFk qowlvVssPtAFW9HtD+XBXuArXCnQlDHuTsoFHS0Wm4oGKi+d7u0mS04qumyvY/iXl2Hr 0lYg== X-Forwarded-Encrypted: i=1; AJvYcCVpBJF/BjHnfrz1cmFVnkd9Z1aegqgK6yX3Z29v+62dd8MrKDR3DBU+wYbdgKN+l4JiFwzQbucWbVgN2E2P8ev0ugAKb5A= X-Gm-Message-State: AOJu0Yz2IQ20k4FKb8QHFtkk5mG0mR7gUSxvTGoxdeD6DZdk7MAi0dcQ FekmDrH0VJlSqvWqWz7VkvyKp0g2cHZz5os7lwdPmMPIEKGgf50v4WOjI9sp X-Google-Smtp-Source: AGHT+IGnNRNYvM+8UNLkK0FjYRjOuGJUt1cOE5VT6xVg74RQuhBKTk0KoAUrYzFQ7q9i+AVio40mkg== X-Received: by 2002:a05:6870:e414:b0:23e:5175:8de2 with SMTP id n20-20020a056870e41400b0023e51758de2mr5238476oag.44.1714812293584; Sat, 04 May 2024 01:44:53 -0700 (PDT) Received: from amjad-pc.. ([192.228.203.229]) by smtp.gmail.com with ESMTPSA id ei32-20020a056a0080e000b006f44e64dfe3sm2310393pfb.177.2024.05.04.01.44.51 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Sat, 04 May 2024 01:44:53 -0700 (PDT) From: Amjad Alsharafi To: qemu-devel@nongnu.org Cc: Hanna Reitz , Kevin Wolf , "open list:vvfat" , Amjad Alsharafi Subject: [PATCH v2 5/5] iotests: Filter out `vvfat` fmt from failing tests Date: Sat, 4 May 2024 16:44:23 +0800 Message-ID: X-Mailer: git-send-email 2.44.0 In-Reply-To: References: MIME-Version: 1.0 Received-SPF: pass client-ip=2001:4860:4864:20::2b; envelope-from=amjadsharafi10@gmail.com; helo=mail-oa1-x2b.google.com X-Spam_score_int: -17 X-Spam_score: -1.8 X-Spam_bar: - X-Spam_report: (-1.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.29 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org `vvfat` is a special format and not all tests (even generic) can run without crashing. So, added `unsupported_fmt: vvfat` to all failling tests. Also added `vvfat` format into `meson.build`, vvfaat tests can be run on the `block-thorough` suite. Signed-off-by: Amjad Alsharafi --- .gitlab-ci.d/buildtest.yml | 1 + tests/qemu-iotests/001 | 1 + tests/qemu-iotests/002 | 1 + tests/qemu-iotests/003 | 1 + tests/qemu-iotests/005 | 1 + tests/qemu-iotests/008 | 1 + tests/qemu-iotests/009 | 1 + tests/qemu-iotests/010 | 1 + tests/qemu-iotests/011 | 1 + tests/qemu-iotests/012 | 1 + tests/qemu-iotests/021 | 1 + tests/qemu-iotests/032 | 1 + tests/qemu-iotests/033 | 1 + tests/qemu-iotests/052 | 1 + tests/qemu-iotests/094 | 1 + tests/qemu-iotests/120 | 2 +- tests/qemu-iotests/140 | 1 + tests/qemu-iotests/145 | 1 + tests/qemu-iotests/157 | 1 + tests/qemu-iotests/159 | 2 +- tests/qemu-iotests/170 | 2 +- tests/qemu-iotests/192 | 1 + tests/qemu-iotests/197 | 2 +- tests/qemu-iotests/208 | 2 +- tests/qemu-iotests/215 | 2 +- tests/qemu-iotests/236 | 2 +- tests/qemu-iotests/251 | 1 + tests/qemu-iotests/307 | 2 +- tests/qemu-iotests/308 | 2 +- tests/qemu-iotests/meson.build | 3 ++- tests/qemu-iotests/tests/export-incoming-iothread | 2 +- tests/qemu-iotests/tests/fuse-allow-other | 1 + tests/qemu-iotests/tests/mirror-ready-cancel-error | 2 +- tests/qemu-iotests/tests/regression-vhdx-log | 1 + 34 files changed, 35 insertions(+), 12 deletions(-) diff --git a/.gitlab-ci.d/buildtest.yml b/.gitlab-ci.d/buildtest.yml index cfdff175c3..a46c179a6b 100644 --- a/.gitlab-ci.d/buildtest.yml +++ b/.gitlab-ci.d/buildtest.yml @@ -347,6 +347,7 @@ build-tcg-disabled: 124 132 139 142 144 145 151 152 155 157 165 194 196 200 202 208 209 216 218 227 234 246 247 248 250 254 255 257 258 260 261 262 263 264 270 272 273 277 279 image-fleecing + - ./check -vvfat vvfat build-user: extends: .native_build_job_template diff --git a/tests/qemu-iotests/001 b/tests/qemu-iotests/001 index 6f980fd34d..cf905b5d00 100755 --- a/tests/qemu-iotests/001 +++ b/tests/qemu-iotests/001 @@ -38,6 +38,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 . ./common.filter _supported_fmt generic +_unsupported_fmt vvfat _supported_proto generic diff --git a/tests/qemu-iotests/002 b/tests/qemu-iotests/002 index 5ce1647531..1e557fad8c 100755 --- a/tests/qemu-iotests/002 +++ b/tests/qemu-iotests/002 @@ -38,6 +38,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 . ./common.filter _supported_fmt generic +_unsupported_fmt vvfat _supported_proto generic _unsupported_imgopts "subformat=streamOptimized" diff --git a/tests/qemu-iotests/003 b/tests/qemu-iotests/003 index 03f902a83c..6e74f1faeb 100755 --- a/tests/qemu-iotests/003 +++ b/tests/qemu-iotests/003 @@ -38,6 +38,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 . ./common.filter _supported_fmt generic +_unsupported_fmt vvfat _supported_proto generic _unsupported_imgopts "subformat=streamOptimized" diff --git a/tests/qemu-iotests/005 b/tests/qemu-iotests/005 index ba377543b0..28ae66bfcd 100755 --- a/tests/qemu-iotests/005 +++ b/tests/qemu-iotests/005 @@ -41,6 +41,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 . ./common.filter _supported_fmt generic +_unsupported_fmt vvfat _supported_proto generic _supported_os Linux _unsupported_imgopts "subformat=twoGbMaxExtentFlat" \ diff --git a/tests/qemu-iotests/008 b/tests/qemu-iotests/008 index fa4990b513..80850ecf12 100755 --- a/tests/qemu-iotests/008 +++ b/tests/qemu-iotests/008 @@ -38,6 +38,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 . ./common.filter _supported_fmt generic +_unsupported_fmt vvfat _supported_proto generic diff --git a/tests/qemu-iotests/009 b/tests/qemu-iotests/009 index efa852bad3..408617b0bc 100755 --- a/tests/qemu-iotests/009 +++ b/tests/qemu-iotests/009 @@ -38,6 +38,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 . ./common.filter _supported_fmt generic +_unsupported_fmt vvfat _supported_proto generic _unsupported_imgopts "subformat=streamOptimized" diff --git a/tests/qemu-iotests/010 b/tests/qemu-iotests/010 index 4ae9027b47..c9f6279255 100755 --- a/tests/qemu-iotests/010 +++ b/tests/qemu-iotests/010 @@ -38,6 +38,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 . ./common.filter _supported_fmt generic +_unsupported_fmt vvfat _supported_proto generic _unsupported_imgopts "subformat=streamOptimized" diff --git a/tests/qemu-iotests/011 b/tests/qemu-iotests/011 index 5c99ac987f..92039fa949 100755 --- a/tests/qemu-iotests/011 +++ b/tests/qemu-iotests/011 @@ -38,6 +38,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 . ./common.filter _supported_fmt generic +_unsupported_fmt vvfat _supported_proto generic _unsupported_imgopts "subformat=streamOptimized" diff --git a/tests/qemu-iotests/012 b/tests/qemu-iotests/012 index 3a24d2ca8d..5b0f1338e6 100755 --- a/tests/qemu-iotests/012 +++ b/tests/qemu-iotests/012 @@ -40,6 +40,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 . ./common.filter _supported_fmt generic +_unsupported_fmt vvfat _supported_proto file # Remove once all tests are fixed to use TEST_IMG_FILE diff --git a/tests/qemu-iotests/021 b/tests/qemu-iotests/021 index 0fc89df2fe..475f9b2116 100755 --- a/tests/qemu-iotests/021 +++ b/tests/qemu-iotests/021 @@ -38,6 +38,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 . ./common.filter _supported_fmt generic +_unsupported_fmt vvfat _supported_proto generic diff --git a/tests/qemu-iotests/032 b/tests/qemu-iotests/032 index ebbe7cb0ba..b58141f132 100755 --- a/tests/qemu-iotests/032 +++ b/tests/qemu-iotests/032 @@ -42,6 +42,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 # This works for any image format (though unlikely to segfault for raw) _supported_fmt generic +_unsupported_fmt vvfat _supported_proto generic _unsupported_imgopts "subformat=streamOptimized" diff --git a/tests/qemu-iotests/033 b/tests/qemu-iotests/033 index 4bc7a071bd..6410c8717e 100755 --- a/tests/qemu-iotests/033 +++ b/tests/qemu-iotests/033 @@ -38,6 +38,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 . ./common.filter _supported_fmt generic +_unsupported_fmt vvfat _supported_proto generic _unsupported_imgopts "subformat=streamOptimized" diff --git a/tests/qemu-iotests/052 b/tests/qemu-iotests/052 index 2f23ac9b65..5b3545c8b9 100755 --- a/tests/qemu-iotests/052 +++ b/tests/qemu-iotests/052 @@ -38,6 +38,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 . ./common.filter _supported_fmt generic +_unsupported_fmt vvfat _supported_proto file # Don't do O_DIRECT on tmpfs diff --git a/tests/qemu-iotests/094 b/tests/qemu-iotests/094 index 4766e9a458..d8da955c1b 100755 --- a/tests/qemu-iotests/094 +++ b/tests/qemu-iotests/094 @@ -42,6 +42,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 . ./common.qemu _supported_fmt generic +_unsupported_fmt vvfat _supported_proto nbd _unsupported_imgopts "subformat=monolithicFlat" "subformat=twoGbMaxExtentFlat" diff --git a/tests/qemu-iotests/120 b/tests/qemu-iotests/120 index ac7bd8c4e3..d8e5f4241a 100755 --- a/tests/qemu-iotests/120 +++ b/tests/qemu-iotests/120 @@ -40,7 +40,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 _supported_fmt generic _supported_proto file fuse -_unsupported_fmt luks +_unsupported_fmt luks vvfat _require_drivers raw _make_test_img 64M diff --git a/tests/qemu-iotests/140 b/tests/qemu-iotests/140 index d923b777e2..42a96d9097 100755 --- a/tests/qemu-iotests/140 +++ b/tests/qemu-iotests/140 @@ -45,6 +45,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 . ./common.qemu _supported_fmt generic +_unsupported_fmt vvfat _supported_proto file fuse _supported_os Linux diff --git a/tests/qemu-iotests/145 b/tests/qemu-iotests/145 index a2ce92516d..ff9c6ff54f 100755 --- a/tests/qemu-iotests/145 +++ b/tests/qemu-iotests/145 @@ -39,6 +39,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 . ./common.filter _supported_fmt generic +_unsupported_fmt vvfat _supported_proto generic _make_test_img 1M diff --git a/tests/qemu-iotests/157 b/tests/qemu-iotests/157 index aa2ebbfb4b..419d3b8b7a 100755 --- a/tests/qemu-iotests/157 +++ b/tests/qemu-iotests/157 @@ -38,6 +38,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 . ./common.filter _supported_fmt generic +_unsupported_fmt vvfat _supported_proto file _require_devices virtio-blk diff --git a/tests/qemu-iotests/159 b/tests/qemu-iotests/159 index 4eb476d3a8..70a1079ae5 100755 --- a/tests/qemu-iotests/159 +++ b/tests/qemu-iotests/159 @@ -39,7 +39,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 _supported_fmt generic _supported_proto file -_unsupported_fmt luks +_unsupported_fmt luks vvfat TEST_SIZES="5 512 1024 1999 1K 64K 1M" diff --git a/tests/qemu-iotests/170 b/tests/qemu-iotests/170 index 41387e4d66..f08fb0e8bd 100755 --- a/tests/qemu-iotests/170 +++ b/tests/qemu-iotests/170 @@ -39,7 +39,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 _supported_fmt generic _supported_proto file -_unsupported_fmt luks +_unsupported_fmt luks vvfat echo echo "== Creating image ==" diff --git a/tests/qemu-iotests/192 b/tests/qemu-iotests/192 index e66e1a4f06..ca72b0b7c8 100755 --- a/tests/qemu-iotests/192 +++ b/tests/qemu-iotests/192 @@ -42,6 +42,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 . ./common.qemu _supported_fmt generic +_unsupported_fmt vvfat _supported_proto file if [ "$QEMU_DEFAULT_MACHINE" != "pc" ]; then diff --git a/tests/qemu-iotests/197 b/tests/qemu-iotests/197 index 69849c800e..76b30672d9 100755 --- a/tests/qemu-iotests/197 +++ b/tests/qemu-iotests/197 @@ -53,7 +53,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 _supported_fmt generic _supported_proto generic # LUKS support may be possible, but it complicates things. -_unsupported_fmt luks +_unsupported_fmt luks vvfat _unsupported_imgopts "subformat=streamOptimized" echo diff --git a/tests/qemu-iotests/208 b/tests/qemu-iotests/208 index 6117f165fa..f08c83c0c1 100755 --- a/tests/qemu-iotests/208 +++ b/tests/qemu-iotests/208 @@ -23,7 +23,7 @@ import iotests -iotests.script_initialize(supported_fmts=['generic']) +iotests.script_initialize(supported_fmts=['generic'], unsupported_fmts=['vvfat']) with iotests.FilePath('disk.img') as disk_img_path, \ iotests.FilePath('disk-snapshot.img') as disk_snapshot_img_path, \ diff --git a/tests/qemu-iotests/215 b/tests/qemu-iotests/215 index 6babbcdc1f..3bd03c741e 100755 --- a/tests/qemu-iotests/215 +++ b/tests/qemu-iotests/215 @@ -50,7 +50,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 _supported_fmt generic _supported_proto generic # LUKS support may be possible, but it complicates things. -_unsupported_fmt luks +_unsupported_fmt luks vvfat _unsupported_imgopts "subformat=streamOptimized" echo diff --git a/tests/qemu-iotests/236 b/tests/qemu-iotests/236 index 20419bbb9e..4bcca355ab 100755 --- a/tests/qemu-iotests/236 +++ b/tests/qemu-iotests/236 @@ -23,7 +23,7 @@ import iotests from iotests import log -iotests.script_initialize(supported_fmts=['generic']) +iotests.script_initialize(supported_fmts=['generic'], unsupported_fmts=['vvfat']) size = 64 * 1024 * 1024 granularity = 64 * 1024 diff --git a/tests/qemu-iotests/251 b/tests/qemu-iotests/251 index 794cad58b2..ac83f69d9a 100755 --- a/tests/qemu-iotests/251 +++ b/tests/qemu-iotests/251 @@ -39,6 +39,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 . ./common.qemu _supported_fmt generic +_unsupported_fmt vvfat _supported_proto file _supported_os Linux _unsupported_imgopts "subformat=streamOptimized" diff --git a/tests/qemu-iotests/307 b/tests/qemu-iotests/307 index b429b5aa50..548d2f040d 100755 --- a/tests/qemu-iotests/307 +++ b/tests/qemu-iotests/307 @@ -27,7 +27,7 @@ import os # luks which requires special command lines) iotests.script_initialize( supported_fmts=['generic'], - unsupported_fmts=['luks', 'vpc'], + unsupported_fmts=['luks', 'vpc', 'vvfat'], supported_platforms=['linux'], ) diff --git a/tests/qemu-iotests/308 b/tests/qemu-iotests/308 index ea81dc496a..993dd2a5f9 100755 --- a/tests/qemu-iotests/308 +++ b/tests/qemu-iotests/308 @@ -47,7 +47,7 @@ if [ "$IMGOPTSSYNTAX" = "true" ]; then fi # We need the image to have exactly the specified size, and VPC does # not allow that by default -_unsupported_fmt vpc +_unsupported_fmt vpc vvfat _supported_proto file # We create the FUSE export manually _supported_os Linux # We need /dev/urandom diff --git a/tests/qemu-iotests/meson.build b/tests/qemu-iotests/meson.build index fad340ad59..e87cf71fc4 100644 --- a/tests/qemu-iotests/meson.build +++ b/tests/qemu-iotests/meson.build @@ -23,7 +23,8 @@ qemu_iotests_formats = { 'raw': 'slow', 'qed': 'thorough', 'vmdk': 'thorough', - 'vpc': 'thorough' + 'vpc': 'thorough', + 'vvfat': 'thorough', } foreach k, v : emulators diff --git a/tests/qemu-iotests/tests/export-incoming-iothread b/tests/qemu-iotests/tests/export-incoming-iothread index d36d6194e0..9535046dfd 100755 --- a/tests/qemu-iotests/tests/export-incoming-iothread +++ b/tests/qemu-iotests/tests/export-incoming-iothread @@ -75,5 +75,5 @@ class TestExportIncomingIothread(iotests.QMPTestCase): if __name__ == '__main__': iotests.main(supported_fmts=['generic'], - unsupported_fmts=['luks'], # Would need a secret + unsupported_fmts=['luks', 'vvfat'], # Would need a secret supported_protocols=['file']) diff --git a/tests/qemu-iotests/tests/fuse-allow-other b/tests/qemu-iotests/tests/fuse-allow-other index 19f494aefb..6cfbe9ef1f 100755 --- a/tests/qemu-iotests/tests/fuse-allow-other +++ b/tests/qemu-iotests/tests/fuse-allow-other @@ -38,6 +38,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15 . ../common.qemu _supported_fmt generic +_unsupported_fmt vvfat _supported_proto file # We create the FUSE export manually diff --git a/tests/qemu-iotests/tests/mirror-ready-cancel-error b/tests/qemu-iotests/tests/mirror-ready-cancel-error index ed2e46447e..3b36764ecb 100755 --- a/tests/qemu-iotests/tests/mirror-ready-cancel-error +++ b/tests/qemu-iotests/tests/mirror-ready-cancel-error @@ -138,5 +138,5 @@ class TestMirrorReadyCancelError(iotests.QMPTestCase): if __name__ == '__main__': # LUKS would require special key-secret handling in add_blockdevs() iotests.main(supported_fmts=['generic'], - unsupported_fmts=['luks'], + unsupported_fmts=['luks', 'vvfat'], supported_protocols=['file']) diff --git a/tests/qemu-iotests/tests/regression-vhdx-log b/tests/qemu-iotests/tests/regression-vhdx-log index ca264e93d6..eb216c27dd 100755 --- a/tests/qemu-iotests/tests/regression-vhdx-log +++ b/tests/qemu-iotests/tests/regression-vhdx-log @@ -40,6 +40,7 @@ cd .. . ./common.filter _supported_fmt generic +_unsupported_fmt vvfat _supported_proto generic _unsupported_imgopts "subformat=streamOptimized"