From patchwork Tue May 23 11:31:05 2017 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Alexey Perevalov X-Patchwork-Id: 765915 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.gnu.org (lists.gnu.org [IPv6:2001:4830:134:3::11]) (using TLSv1 with cipher AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3wXDcP2DC9z9sPH for ; Tue, 23 May 2017 22:00:21 +1000 (AEST) Received: from localhost ([::1]:48037 helo=lists.gnu.org) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dD8U7-000433-1P for incoming@patchwork.ozlabs.org; Tue, 23 May 2017 08:00:19 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:48799) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from ) id 1dD82L-0006mG-Qy for qemu-devel@nongnu.org; Tue, 23 May 2017 07:31:41 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from ) id 1dD82H-0003gA-SG for qemu-devel@nongnu.org; Tue, 23 May 2017 07:31:37 -0400 Received: from mailout2.w1.samsung.com ([210.118.77.12]:43229) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from ) id 1dD82H-0003er-L1 for qemu-devel@nongnu.org; Tue, 23 May 2017 07:31:33 -0400 Received: from eucas1p2.samsung.com (unknown [182.198.249.207]) by mailout2.w1.samsung.com (Oracle Communications Messaging Server 7.0.5.31.0 64bit (built May 5 2014)) with ESMTP id <0OQE0013QLCHCV70@mailout2.w1.samsung.com> for qemu-devel@nongnu.org; Tue, 23 May 2017 12:31:29 +0100 (BST) Received: from eusmges4.samsung.com (unknown [203.254.199.244]) by eucas1p2.samsung.com (KnoxPortal) with ESMTP id 20170523113129eucas1p2e7334beca347fd2cb237b7e6fa08df21~BOOC4f_bV1787917879eucas1p2T; Tue, 23 May 2017 11:31:29 +0000 (GMT) Received: from eucas1p2.samsung.com ( [182.198.249.207]) by eusmges4.samsung.com (EUCPMTA) with SMTP id EF.C2.04729.09D14295; Tue, 23 May 2017 12:31:28 +0100 (BST) Received: from eusmgms1.samsung.com (unknown [182.198.249.179]) by eucas1p1.samsung.com (KnoxPortal) with ESMTP id 20170523113128eucas1p17a89f8cb47d5731c50f94c3218ba155f~BOOCSG3fv0479504795eucas1p1_; Tue, 23 May 2017 11:31:28 +0000 (GMT) X-AuditID: cbfec7f4-f79806d000001279-b5-59241d90adb6 Received: from eusync3.samsung.com ( [203.254.199.213]) by eusmgms1.samsung.com (EUCPMTA) with SMTP id 2B.AB.17452.09D14295; Tue, 23 May 2017 12:31:28 +0100 (BST) Received: from aperevalov-ubuntu.rnd.samsung.ru ([106.109.129.199]) by eusync3.samsung.com (Oracle Communications Messaging Server 7.0.5.31.0 64bit (built May 5 2014)) with ESMTPA id <0OQE003QBLC1VWB0@eusync3.samsung.com>; Tue, 23 May 2017 12:31:28 +0100 (BST) From: Alexey Perevalov To: qemu-devel@nongnu.org Date: Tue, 23 May 2017 14:31:05 +0300 Message-id: <1495539071-12995-5-git-send-email-a.perevalov@samsung.com> X-Mailer: git-send-email 1.9.1 In-reply-to: <1495539071-12995-1-git-send-email-a.perevalov@samsung.com> X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFtrNIsWRmVeSWpSXmKPExsWy7djP87oTZFUiDdovGlrMvXuexaJ32z12 i4lv17NaXGn/yW6xZf83dovjvTtYHNg8nlzbzOTxft9VNo++LasYA5ijuGxSUnMyy1KL9O0S uDJ6thxlLFinVbG4dRtLA+MO+S5GTg4JAROJeYdaWSFsMYkL99azdTFycQgJLGWU6H+3kAXC +cwo8f/0aiaYjicTvjCC2EICyxgl/sDY3UwSZ+a7dTFycLAJGEjsu2cLEhYRkJT43XWaGWQO s0Azo8Shxz+ZQRLCAokSl2ceBdvMIqAq0brsPzuIzSvgLrFhzRQ2iF1yEiePTQar4RTwkHh3 +B7YQRIC99kkvh7aywqyTEJAVmLTAWaIeheJfYcfQdnCEq+Ob2GHsGUkOjsOMkH0tjNKdO/s ZIVwJjBKnJn+F6rKXuLUzatgXzIL8ElM2jadGWIBr0RHmxBEiYfEgUlTocodJdZc38oKCaHZ jBJfT61jm8Aos4CRYRWjSGppcW56arGJXnFibnFpXrpecn7uJkZglJ7+d/zLDsbFx6wOMQpw MCrx8Go9VooUYk0sK67MPcQowcGsJMLrI6QSKcSbklhZlVqUH19UmpNafIhRmoNFSZyX69S1 CCGB9MSS1OzU1ILUIpgsEwenVANjxOWw6U9/LGpVm7cxbYqfpL99LseBl9OS5jV27NpTlZqz 5afPqk0zq1tKjgWuqRFUMeH8s03UR8jC+Y1g1e1Pfv+qz7nV3V5zdHHhy/P5EV1rJ4n/4b9y 7TIfd3rt7cj/uguLdx3bYHxLuH+tlY3V+ZKak6euBR54oNf8usLYdf+TjxqTlz75pcRSnJFo qMVcVJwIABbdsD3OAgAA X-Brightmail-Tracker: H4sIAAAAAAAAA+NgFrrFLMWRmVeSWpSXmKPExsVy+t/xq7oTZFUiDSZfkLSYe/c8i0Xvtnvs FhPfrme1uNL+k91iy/5v7BbHe3ewOLB5PLm2mcnj/b6rbB59W1YxBjBHudlkpCampBYppOYl 56dk5qXbKoWGuOlaKCnkJeam2ipF6PqGBCkplCXmlAJ5RgZowME5wD1YSd8uwS2jZ8tRxoJ1 WhWLW7exNDDukO9i5OSQEDCReDLhCyOELSZx4d56NhBbSGAJo8TLiaxdjFxAdi+TxMxpq4AS HBxsAgYS++7ZgtSICEhK/O46zQxSwyzQzCjRuroNrFlYIFHi8syjrCA2i4CqROuy/+wgNq+A u8SGNVPYIJbJSZw8NhmshlPAQ+Ld4XssEIvdJU5PvM46gZF3ASPDKkaR1NLi3PTcYkO94sTc 4tK8dL3k/NxNjMBw3Xbs5+YdjJc2Bh9iFOBgVOLh1XisFCnEmlhWXJl7iFGCg1lJhNdHSCVS iDclsbIqtSg/vqg0J7X4EKMp0FETmaVEk/OBsZRXEm9oYmhuaWhkbGFhbmSkJM5b8uFKuJBA emJJanZqakFqEUwfEwenVAPj1AyTEJbJ0bEl711+VljdFOMQXbugRz5wZql4tPibf/wFTXqC ika5lmsljinc3Nc++/LT+dd49LX/PcmNiptkOGfvufLN3zy3lnDWuFw/aj+tnvnnyaObls1b ZvTPVac9aZqb7f6P5porJxmrlOx2bmzwSdi5s15rc8v1SOMmcyn58IRj+wqUWIozEg21mIuK EwEkYa3obQIAAA== X-MTR: 20000000000000000@CPGS X-CMS-MailID: 20170523113128eucas1p17a89f8cb47d5731c50f94c3218ba155f X-Msg-Generator: CA X-Sender-IP: 182.198.249.179 X-Local-Sender: =?UTF-8?B?QWxleGV5IFBlcmV2YWxvdhtTUlItVmlydHVhbGl6YXRpb24g?= =?UTF-8?B?TGFiG+yCvOyEseyghOyekBtTZW5pb3IgRW5naW5lZXI=?= X-Global-Sender: =?UTF-8?B?QWxleGV5IFBlcmV2YWxvdhtTUlItVmlydHVhbGl6YXRpb24g?= =?UTF-8?B?TGFiG1NhbXN1bmcgRWxlY3Ryb25pY3MbU2VuaW9yIEVuZ2luZWVy?= X-Sender-Code: =?UTF-8?B?QzEwG0NJU0hRG0MxMEdEMDFHRDAxMDE1NA==?= CMS-TYPE: 201P X-HopCount: 7 X-CMS-RootMailID: 20170523113128eucas1p17a89f8cb47d5731c50f94c3218ba155f X-RootMTR: 20170523113128eucas1p17a89f8cb47d5731c50f94c3218ba155f References: <1495539071-12995-1-git-send-email-a.perevalov@samsung.com> X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 210.118.77.12 Subject: [Qemu-devel] [PATCH V6 04/10] migration: split ufd_version_check onto receive/request features part X-BeenThere: qemu-devel@nongnu.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: i.maximets@samsung.com, dgilbert@redhat.com, peterx@redhat.com, Alexey Perevalov Errors-To: qemu-devel-bounces+incoming=patchwork.ozlabs.org@nongnu.org Sender: "Qemu-devel" This modification is necessary for userfault fd features which are required to be requested from userspace. UFFD_FEATURE_THREAD_ID is a one of such "on demand" feature, which will be introduced in the next patch. QEMU have to use separate userfault file descriptor, due to userfault context has internal state, and after first call of ioctl UFFD_API it changes its state to UFFD_STATE_RUNNING (in case of success), but kernel while handling ioctl UFFD_API expects UFFD_STATE_WAIT_API. So only one ioctl with UFFD_API is possible per ufd. Signed-off-by: Alexey Perevalov --- migration/postcopy-ram.c | 100 ++++++++++++++++++++++++++++++++++++++++++----- 1 file changed, 91 insertions(+), 9 deletions(-) diff --git a/migration/postcopy-ram.c b/migration/postcopy-ram.c index 3ed78bf..4f3f495 100644 --- a/migration/postcopy-ram.c +++ b/migration/postcopy-ram.c @@ -59,32 +59,114 @@ struct PostcopyDiscardState { #include #include -static bool ufd_version_check(int ufd, MigrationIncomingState *mis) + +/** + * receive_ufd_features: check userfault fd features, to request only supported + * features in the future. + * + * Returns: true on success + * + * __NR_userfaultfd - should be checked before + * @features: out parameter will contain uffdio_api.features provided by kernel + * in case of success + */ +static bool receive_ufd_features(uint64_t *features) { - struct uffdio_api api_struct; - uint64_t ioctl_mask; + struct uffdio_api api_struct = {0}; + int ufd; + bool ret = true; + + /* if we are here __NR_userfaultfd should exists */ + ufd = syscall(__NR_userfaultfd, O_CLOEXEC); + if (ufd == -1) { + error_report("%s: syscall __NR_userfaultfd failed: %s", __func__, + strerror(errno)); + return false; + } + /* ask features */ api_struct.api = UFFD_API; api_struct.features = 0; if (ioctl(ufd, UFFDIO_API, &api_struct)) { - error_report("%s: UFFDIO_API failed: %s", __func__ + error_report("%s: UFFDIO_API failed: %s", __func__, strerror(errno)); + ret = false; + goto release_ufd; + } + + *features = api_struct.features; + +release_ufd: + close(ufd); + return ret; +} + +/** + * request_ufd_features: this function should be called only once on a newly + * opened ufd, subsequent calls will lead to error. + * + * Returns: true on succes + * + * @ufd: fd obtained from userfaultfd syscall + * @features: bit mask see UFFD_API_FEATURES + */ +static bool request_ufd_features(int ufd, uint64_t features) +{ + struct uffdio_api api_struct = {0}; + uint64_t ioctl_mask; + + api_struct.api = UFFD_API; + api_struct.features = features; + if (ioctl(ufd, UFFDIO_API, &api_struct)) { + error_report("%s failed: UFFDIO_API failed: %s", __func__, + strerror(errno)); return false; } - ioctl_mask = (__u64)1 << _UFFDIO_REGISTER | - (__u64)1 << _UFFDIO_UNREGISTER; + ioctl_mask = 1 << _UFFDIO_REGISTER | + 1 << _UFFDIO_UNREGISTER; if ((api_struct.ioctls & ioctl_mask) != ioctl_mask) { error_report("Missing userfault features: %" PRIx64, (uint64_t)(~api_struct.ioctls & ioctl_mask)); return false; } + return true; +} + +static bool ufd_check_and_apply(int ufd, MigrationIncomingState *mis) +{ + uint64_t asked_features = 0; + static uint64_t supported_features; + + /* + * it's not possible to + * request UFFD_API twice per one fd + * userfault fd features is persistent + */ + if (!supported_features) { + if (!receive_ufd_features(&supported_features)) { + error_report("%s failed", __func__); + return false; + } + } + + /* + * request features, even if asked_features is 0, due to + * kernel expects UFFD_API before UFFDIO_REGISTER, per + * userfault file descriptor + */ + if (!request_ufd_features(ufd, asked_features)) { + error_report("%s failed: features %" PRIu64, __func__, + asked_features); + return false; + } + if (getpagesize() != ram_pagesize_summary()) { bool have_hp = false; /* We've got a huge page */ #ifdef UFFD_FEATURE_MISSING_HUGETLBFS - have_hp = api_struct.features & UFFD_FEATURE_MISSING_HUGETLBFS; + have_hp = supported_features & UFFD_FEATURE_MISSING_HUGETLBFS; #endif if (!have_hp) { error_report("Userfault on this host does not support huge pages"); @@ -135,7 +217,7 @@ bool postcopy_ram_supported_by_host(MigrationIncomingState *mis) } /* Version and features check */ - if (!ufd_version_check(ufd, mis)) { + if (!ufd_check_and_apply(ufd, mis)) { goto out; } @@ -512,7 +594,7 @@ int postcopy_ram_enable_notify(MigrationIncomingState *mis) * Although the host check already tested the API, we need to * do the check again as an ABI handshake on the new fd. */ - if (!ufd_version_check(mis->userfault_fd, mis)) { + if (!ufd_check_and_apply(mis->userfault_fd, mis)) { return -1; }