From patchwork Tue Oct 23 23:57:36 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Brendan Higgins X-Patchwork-Id: 988420 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=lists.infradead.org (client-ip=2607:7c80:54:e::133; helo=bombadil.infradead.org; envelope-from=linux-um-bounces+incoming=patchwork.ozlabs.org@lists.infradead.org; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=reject dis=none) header.from=google.com Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="ItiZ1y+s"; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=google.com header.i=@google.com header.b="Z7BqesJk"; dkim-atps=neutral Received: from bombadil.infradead.org (bombadil.infradead.org [IPv6:2607:7c80:54:e::133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 42fr2f5cbmz9s4s for ; Wed, 24 Oct 2018 10:59:06 +1100 (AEDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:Cc:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:To:From:Subject:Mime-Version:Message-Id :Date:Reply-To:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References: List-Owner; bh=S5u4DC1PVsoSyPtESEWZZo4LOhhMfMcVlYdc8i0rd0E=; b=ItiZ1y+smVD4tH eZtt3Au3QXhg58tYpmchlwH8zdIzTMqbE2S2M+sjoCoedrCTVreRrkI8DgWY9OMAvi2s/1kjJ2C3y EI9Ii8Z3mHeu/Je5z8L8gN3B9VouSzsE+b9vVKAhdNVPCxcZXXkv4Q6UCuVccMUZvzhpnHpuDljig V+1JgPOmHLF1yhDM0R0nUMtIWabtf/IsRirhnqWo0sxQdj19yxbRGFV+N8TbPbLQpb99PoOYJReX3 eVPM8QH93Ml7/FXfUZ7HRZ0RxHGJ9SowsLwp/VluD+hK2RRwUNoQuJaVFmss1Te8m7QxEwRfW/J6D nrYzZp7dg0Tf5k6t+U1A==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.90_1 #2 (Red Hat Linux)) id 1gF6Zf-00041B-DP; Tue, 23 Oct 2018 23:58:59 +0000 Received: from mail-qt1-x849.google.com ([2607:f8b0:4864:20::849]) by bombadil.infradead.org with esmtps (Exim 4.90_1 #2 (Red Hat Linux)) id 1gF6ZZ-0003xZ-R5 for linux-um@lists.infradead.org; Tue, 23 Oct 2018 23:58:56 +0000 Received: by mail-qt1-x849.google.com with SMTP id j60-v6so3778780qtb.8 for ; Tue, 23 Oct 2018 16:58:43 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20161025; h=date:message-id:mime-version:subject:from:to:cc; bh=ZS2dPkLUDF5rzLuADBdbF55GgpLIevIetdCiSG5eHMw=; b=Z7BqesJk5QdbsF7zt5bRUxTXdE3EMzjlx4YZ5rYhcNVYjhIZjSikzed3mJgg1xazPy 2IhRQlNxnlcddLtGdFKWFr3354ZqRbVH0608EijX2B5ABitG2orNV1s0pwDgEbLKVaw8 OqrqbzKubtOP9YwLlCcUsf/R5QKeZI9wVm+HnCg5omP+KSP/factrtUpKyJOTj3RkSyM GBgpl6qnzZofSzYSI70PB69fk3BJC4vWHkKNn5nYZs9gTifR9BNi7DCp4HADDsNOs2PN dhowm35ZYvw/ie6OJOgFu6T6/ablpfb++1aTBJmV/GVM/AQB/Ll5Go1dxN/xdeIRpUrD Ul6Q== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:date:message-id:mime-version:subject:from:to:cc; bh=ZS2dPkLUDF5rzLuADBdbF55GgpLIevIetdCiSG5eHMw=; b=iEUM2xtu9TYsg7ZE9EhL8ansPyVJcXPa60fwmo1XxI9C7cFmiS8PKNjkU9YGmyAi37 dOG74DMvAhmLDJxsW6af/Et4ywd20ylH8UbNxsbROjFLVtpTTTVk8KGzKBQ9CEUy6BY5 XmAcpEqrDUsghRpDbU8rPGjzmPiLrWOGrXVyKfhAIEu/yET3tS72AeU5aA54GHiSHq/e UKMeedNLw/bqTJVYh3ULFoV1XpTKDMHFntmdBN4Nkkdog53HrAdkauTh1Dn+E5KWROpc xpXaoWfUR/0heNOwUlZ9RCSj3fz51KlgtC7IV/TlRuLkGz7GjLkolFOavAhPqcrRXcdN gK8g== X-Gm-Message-State: AGRZ1gKODBO2gNT2Re2nY3uRybIXTpqhpHdbenc9yPQh0ofh1XWFPv5N yTiEbzffyjbqAcC3IsomM1Ln9EuKMWTmF/VKYa34lw== X-Google-Smtp-Source: AJdET5e72o3fX8sjy4I2lSUuxpUYKL7PmhaEGhr21zADJYN/VeaMFb5QSPJNWQmQ9FQZ/HRbEAXB3Hg7uI15O3w/xBrKAQ== X-Received: by 2002:a0c:9e2e:: with SMTP id p46mr338716qve.22.1540339122147; Tue, 23 Oct 2018 16:58:42 -0700 (PDT) Date: Tue, 23 Oct 2018 16:57:36 -0700 Message-Id: <20181023235750.103146-1-brendanhiggins@google.com> Mime-Version: 1.0 X-Mailer: git-send-email 2.19.1.568.g152ad8e336-goog Subject: [RFC v2 00/14] kunit: introduce KUnit, the Linux kernel unit testing framework From: Brendan Higgins To: gregkh@linuxfoundation.org, keescook@google.com, mcgrof@kernel.org, shuah@kernel.org X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20181023_165853_940567_8D004B0A X-CRM114-Status: GOOD ( 10.12 ) X-Spam-Score: -7.6 (-------) X-Spam-Report: SpamAssassin version 3.4.1 on bombadil.infradead.org summary: Content analysis details: (-7.6 points) pts rule name description ---- ---------------------- -------------------------------------------------- -0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at http://www.dnswl.org/, no trust [2607:f8b0:4864:20:0:0:0:849 listed in] [list.dnswl.org] -0.0 SPF_PASS SPF: sender matches SPF record -7.5 USER_IN_DEF_DKIM_WL From: address is in the default DKIM white-list 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature -0.0 DKIMWL_WL_MED DKIMwl.org - Whitelisted Medium sender X-BeenThere: linux-um@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: brakmo@fb.com, robh@kernel.org, richard@nod.at, dri-devel@lists.freedesktop.org, linux-nvdimm@lists.01.org, mpe@ellerman.id.au, Tim.Bird@sony.com, linux-um@lists.infradead.org, linux-kernel@vger.kernel.org, rostedt@goodmis.org, kieran.bingham@ideasonboard.com, julia.lawall@lip6.fr, joel@jms.id.au, linux-kselftest@vger.kernel.org, khilman@baylibre.com, joe@perches.com, daniel@ffwll.ch, dan.j.williams@intel.com, jdike@addtoit.com, Brendan Higgins , kunit-dev@googlegroups.com Sender: "linux-um" Errors-To: linux-um-bounces+incoming=patchwork.ozlabs.org@lists.infradead.org This patch set proposes KUnit, a lightweight unit testing and mocking framework for the Linux kernel. Unlike Autotest and kselftest, KUnit is a true unit testing framework; it does not require installing the kernel on a test machine or in a VM and does not require tests to be written in userspace running on a host kernel. Additionally, KUnit is fast: From invocation to completion KUnit can run several dozen tests in under a second. Currently, the entire KUnit test suite for KUnit runs in under a second from the initial invocation (build time excluded). KUnit is heavily inspired by JUnit, Python's unittest.mock, and Googletest/Googlemock for C++. KUnit provides facilities for defining unit test cases, grouping related test cases into test suites, providing common infrastructure for running tests, mocking, spying, and much more. ## What's so special about unit testing? A unit test is supposed to test a single unit of code in isolation, hence the name. There should be no dependencies outside the control of the test; this means no external dependencies, which makes tests orders of magnitudes faster. Likewise, since there are no external dependencies, there are no hoops to jump through to run the tests. Additionally, this makes unit tests deterministic: a failing unit test always indicates a problem. Finally, because unit tests necessarily have finer granularity, they are able to test all code paths easily solving the classic problem of difficulty in exercising error handling code. ## Is KUnit trying to replace other testing frameworks for the kernel? No. Most existing tests for the Linux kernel are end-to-end tests, which have their place. A well tested system has lots of unit tests, a reasonable number of integration tests, and some end-to-end tests. KUnit is just trying to address the unit test space which is currently not being addressed. ## More information on KUnit There is a bunch of documentation near the end of this patch set that describes how to use KUnit and best practices for writing unit tests. For convenience I am hosting the compiled docs here: https://google.github.io/kunit-docs/third_party/kernel/docs/ ## Changes Since Last Version - Updated patchset to apply cleanly on 4.19. - Stripped down patchset to focus on just the core features (I dropped mocking, spying, and the MMIO stuff for now; you can find these patches here: https://kunit-review.googlesource.com/c/linux/+/1132), as suggested by Rob. - Cleaned up some of the commit messages and tweaked commit order a bit based on suggestions.