From patchwork Mon May 7 22:18:23 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Joseph Kogut X-Patchwork-Id: 909962 Return-Path: X-Original-To: incoming-buildroot@patchwork.ozlabs.org Delivered-To: patchwork-incoming-buildroot@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=busybox.net (client-ip=140.211.166.137; helo=fraxinus.osuosl.org; envelope-from=buildroot-bounces@busybox.net; receiver=) Authentication-Results: ozlabs.org; dmarc=fail (p=none dis=none) header.from=gmail.com Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b="oUTe+n3F"; dkim-atps=neutral Received: from fraxinus.osuosl.org (smtp4.osuosl.org [140.211.166.137]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 40fxpm4DkPz9s27 for ; Tue, 8 May 2018 08:18:40 +1000 (AEST) Received: from localhost (localhost [127.0.0.1]) by fraxinus.osuosl.org (Postfix) with ESMTP id A21E185A92; Mon, 7 May 2018 22:18:37 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from fraxinus.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id K2zMLRhvBJZ1; Mon, 7 May 2018 22:18:36 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by fraxinus.osuosl.org (Postfix) with ESMTP id 67A7985A75; Mon, 7 May 2018 22:18:36 +0000 (UTC) X-Original-To: buildroot@lists.busybox.net Delivered-To: buildroot@osuosl.org Received: from hemlock.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by ash.osuosl.org (Postfix) with ESMTP id DA4561C1F90 for ; Mon, 7 May 2018 22:18:34 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by hemlock.osuosl.org (Postfix) with ESMTP id D77E786E86 for ; Mon, 7 May 2018 22:18:34 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from hemlock.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id EtCdbBz9qSnm for ; Mon, 7 May 2018 22:18:33 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mail-pl0-f65.google.com (mail-pl0-f65.google.com [209.85.160.65]) by hemlock.osuosl.org (Postfix) with ESMTPS id 77C6886E71 for ; Mon, 7 May 2018 22:18:33 +0000 (UTC) Received: by mail-pl0-f65.google.com with SMTP id ay10-v6so960085plb.1 for ; Mon, 07 May 2018 15:18:33 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=stKI2sRhTYPNIouzJTK4oVAzBUjPO8ELxSpYAdaQNBE=; b=oUTe+n3FptcLPZjojb1ls2xSuWpb+uN/DaX44R0+0djZuOSUL3G+nDBFf0JWZ4RzWN whe3If5ENHRVWag9BDHtSUl5GpgzPebvWRhpDupFtK1tqXtusagrO0fPg6HVkwn4K1CH Es4zEEuh7Cetn9k+uo93/ZDcuCdRrM1+FC9Nvs2d2cjCvU83zJu8sibTisnYS+FTCn5Z uhegUOYc5LV4HCaNEJbAi045mCoh2cfNR4ZwM2422i3Vidr6Mf8e3dOpMLT/xsD6n9RW DS6M0uikTvmsVlkC6mZQXgIy4MT1C9DsLDJApYZjd394R3SMc16G8jswqggRNqvxPMed 38Vg== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to :references; bh=stKI2sRhTYPNIouzJTK4oVAzBUjPO8ELxSpYAdaQNBE=; b=Ox0APn6jsflyrKM48S7+F0agpdqfTVQ7TwJXqdfRmi6RrhSWmp1vD124VtRteXDZ57 jQdhRaQ+ocglGxS+jB/Xb6U26lQrv+SQIPuCFOPW1CgRjQjt+DpvuWEbqxS97V9F78k+ 1rWo9pOvuazJXwJdVv5bmmTDOWcHc1RV1Ya+ibMKfsGR3L/1k6pB3N0d/afLU3RLKumU EHhjiMvJXOBFnMlH29PxY+AEOegrWKPRZ5HNJYFiY6iIMqz5eNEjagYnPvxXtUeriRfj Mmf6vkvN+GOboOfBzUzv8ym8cBiNSN4Uc47JF0ux4yEqecpTpx3YdUfwtW1xXLdIg6qa ZVag== X-Gm-Message-State: ALQs6tDjsOSgrwqh3ze0pVkBE78DNPw+8pLy2rBf4ObRHItJe16F6eZ3 ZNNf0K8itFelFWtubeLSJhxVq9rm X-Google-Smtp-Source: AB8JxZpzKgRD2MCjq/HB5tVuFVqfQH0H6KCW3B3R9Da2EhO3/wPx4PpJLIgNAWUgh0wAKC2f2ivnOg== X-Received: by 2002:a17:902:2f43:: with SMTP id s61-v6mr38140837plb.99.1525731512552; Mon, 07 May 2018 15:18:32 -0700 (PDT) Received: from JAKWS.YakimaSteel.local ([162.251.69.147]) by smtp.gmail.com with ESMTPSA id u70sm46220059pfk.121.2018.05.07.15.18.31 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Mon, 07 May 2018 15:18:32 -0700 (PDT) From: Joseph Kogut To: buildroot@buildroot.org Date: Mon, 7 May 2018 15:18:23 -0700 Message-Id: <20180507221823.6364-1-joseph.kogut@gmail.com> X-Mailer: git-send-email 2.17.0 In-Reply-To: <20180507205733.2433-1-joseph.kogut@gmail.com> References: <20180507205733.2433-1-joseph.kogut@gmail.com> Subject: [Buildroot] [PATCH v2 1/1] python-websockets: backport fix for upstream issue #350 X-BeenThere: buildroot@busybox.net X-Mailman-Version: 2.1.24 Precedence: list List-Id: Discussion and development of buildroot List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: Joseph Kogut MIME-Version: 1.0 Errors-To: buildroot-bounces@busybox.net Sender: "buildroot" Fetch from: https://github.com/aaugustin/websockets/commit/402059e4a46a764632eba8a669f5b012f173ee7b.patch Signed-off-by: Joseph Kogut --- Changes v1 -> v2: - Include patch locally (suggested by Thomas) ...59e4a46a764632eba8a669f5b012f173ee7b.patch | 256 ++++++++++++++++++ 1 file changed, 256 insertions(+) create mode 100644 package/python-websockets/402059e4a46a764632eba8a669f5b012f173ee7b.patch -- 2.17.0 diff --git a/package/python-websockets/402059e4a46a764632eba8a669f5b012f173ee7b.patch b/package/python-websockets/402059e4a46a764632eba8a669f5b012f173ee7b.patch new file mode 100644 index 0000000000..62e93bdd2e --- /dev/null +++ b/package/python-websockets/402059e4a46a764632eba8a669f5b012f173ee7b.patch @@ -0,0 +1,256 @@ +From 402059e4a46a764632eba8a669f5b012f173ee7b Mon Sep 17 00:00:00 2001 +From: Aymeric Augustin +Date: Tue, 1 May 2018 17:05:05 +0200 +Subject: [PATCH] Fix behavior of recv() in the CLOSING state. + +The behavior wasn't tested correctly: in some test cases, the connection +had already moved to the CLOSED state, where the close code and reason +are already known. + +Refactor half_close_connection_{local,remote} to allow multiple runs of +the event loop while remaining in the CLOSING state. Refactor affected +tests accordingly. + +I verified that all tests in the CLOSING state were behaving is intended +by inserting debug statements in recv/send/ping/pong and running: + +$ PYTHONASYNCIODEBUG=1 python -m unittest -v websockets.test_protocol.{Client,Server}Tests.test_{recv,send,ping,pong}_on_closing_connection_{local,remote} + +Fix #317, #327, #350, #357. +--- + websockets/protocol.py | 10 +++--- + websockets/test_protocol.py | 78 +++++++++++++++++++++++++++++++++++---------- + 2 files changed, 66 insertions(+), 22 deletions(-) + +diff --git a/websockets/protocol.py b/websockets/protocol.py +index f8121a1..7583fe9 100644 +--- a/websockets/protocol.py ++++ b/websockets/protocol.py +@@ -303,7 +303,7 @@ def recv(self): + # Don't yield from self.ensure_open() here because messages could be + # received before the closing frame even if the connection is closing. + +- # Wait for a message until the connection is closed ++ # Wait for a message until the connection is closed. + next_message = asyncio_ensure_future( + self.messages.get(), loop=self.loop) + try: +@@ -315,15 +315,15 @@ def recv(self): + next_message.cancel() + raise + +- # Now there's no need to yield from self.ensure_open(). Either a +- # message was received or the connection was closed. +- + if next_message in done: + return next_message.result() + else: + next_message.cancel() + if not self.legacy_recv: +- raise ConnectionClosed(self.close_code, self.close_reason) ++ assert self.state in [State.CLOSING, State.CLOSED] ++ # Wait until the connection is closed to raise ++ # ConnectionClosed with the correct code and reason. ++ yield from self.ensure_open() + + @asyncio.coroutine + def send(self, data): +diff --git a/websockets/test_protocol.py b/websockets/test_protocol.py +index 70348fb..bfd4e3b 100644 +--- a/websockets/test_protocol.py ++++ b/websockets/test_protocol.py +@@ -105,7 +105,7 @@ def run_loop_once(self): + self.loop.call_soon(self.loop.stop) + self.loop.run_forever() + +- def make_drain_slow(self, delay=3 * MS): ++ def make_drain_slow(self, delay=MS): + # Process connection_made in order to initialize self.protocol.writer. + self.run_loop_once() + +@@ -174,6 +174,8 @@ def close_connection(self, code=1000, reason='close'): + # Empty the outgoing data stream so we can make assertions later on. + self.assertOneFrameSent(True, OP_CLOSE, close_frame_data) + ++ assert self.protocol.state is State.CLOSED ++ + def half_close_connection_local(self, code=1000, reason='close'): + """ + Start a closing handshake but do not complete it. +@@ -181,31 +183,56 @@ def half_close_connection_local(self, code=1000, reason='close'): + The main difference with `close_connection` is that the connection is + left in the CLOSING state until the event loop runs again. + ++ The current implementation returns a task that must be awaited or ++ cancelled, else asyncio complains about destroying a pending task. ++ + """ + close_frame_data = serialize_close(code, reason) +- # Trigger the closing handshake from the local side. +- self.ensure_future(self.protocol.close(code, reason)) ++ # Trigger the closing handshake from the local endpoint. ++ close_task = self.ensure_future(self.protocol.close(code, reason)) + self.run_loop_once() # wait_for executes + self.run_loop_once() # write_frame executes + # Empty the outgoing data stream so we can make assertions later on. + self.assertOneFrameSent(True, OP_CLOSE, close_frame_data) +- # Prepare the response to the closing handshake from the remote side. +- self.loop.call_soon( +- self.receive_frame, Frame(True, OP_CLOSE, close_frame_data)) +- self.loop.call_soon(self.receive_eof_if_client) ++ ++ assert self.protocol.state is State.CLOSING ++ ++ # Complete the closing sequence at 1ms intervals so the test can run ++ # at each point even it goes back to the event loop several times. ++ self.loop.call_later( ++ MS, self.receive_frame, Frame(True, OP_CLOSE, close_frame_data)) ++ self.loop.call_later(2 * MS, self.receive_eof_if_client) ++ ++ # This task must be awaited or cancelled by the caller. ++ return close_task + + def half_close_connection_remote(self, code=1000, reason='close'): + """ +- Receive a closing handshake. ++ Receive a closing handshake but do not complete it. + + The main difference with `close_connection` is that the connection is + left in the CLOSING state until the event loop runs again. + + """ ++ # On the server side, websockets completes the closing handshake and ++ # closes the TCP connection immediately. Yield to the event loop after ++ # sending the close frame to run the test while the connection is in ++ # the CLOSING state. ++ if not self.protocol.is_client: ++ self.make_drain_slow() ++ + close_frame_data = serialize_close(code, reason) +- # Trigger the closing handshake from the remote side. ++ # Trigger the closing handshake from the remote endpoint. + self.receive_frame(Frame(True, OP_CLOSE, close_frame_data)) +- self.receive_eof_if_client() ++ self.run_loop_once() # read_frame executes ++ # Empty the outgoing data stream so we can make assertions later on. ++ self.assertOneFrameSent(True, OP_CLOSE, close_frame_data) ++ ++ assert self.protocol.state is State.CLOSING ++ ++ # Complete the closing sequence at 1ms intervals so the test can run ++ # at each point even it goes back to the event loop several times. ++ self.loop.call_later(2 * MS, self.receive_eof_if_client) + + def process_invalid_frames(self): + """ +@@ -335,11 +362,13 @@ def test_recv_binary(self): + self.assertEqual(data, b'tea') + + def test_recv_on_closing_connection_local(self): +- self.half_close_connection_local() ++ close_task = self.half_close_connection_local() + + with self.assertRaises(ConnectionClosed): + self.loop.run_until_complete(self.protocol.recv()) + ++ self.loop.run_until_complete(close_task) # cleanup ++ + def test_recv_on_closing_connection_remote(self): + self.half_close_connection_remote() + +@@ -421,24 +450,29 @@ def test_send_type_error(self): + self.assertNoFrameSent() + + def test_send_on_closing_connection_local(self): +- self.half_close_connection_local() ++ close_task = self.half_close_connection_local() + + with self.assertRaises(ConnectionClosed): + self.loop.run_until_complete(self.protocol.send('foobar')) ++ + self.assertNoFrameSent() + ++ self.loop.run_until_complete(close_task) # cleanup ++ + def test_send_on_closing_connection_remote(self): + self.half_close_connection_remote() + + with self.assertRaises(ConnectionClosed): + self.loop.run_until_complete(self.protocol.send('foobar')) +- self.assertOneFrameSent(True, OP_CLOSE, serialize_close(1000, 'close')) ++ ++ self.assertNoFrameSent() + + def test_send_on_closed_connection(self): + self.close_connection() + + with self.assertRaises(ConnectionClosed): + self.loop.run_until_complete(self.protocol.send('foobar')) ++ + self.assertNoFrameSent() + + # Test the ping coroutine. +@@ -466,24 +500,29 @@ def test_ping_type_error(self): + self.assertNoFrameSent() + + def test_ping_on_closing_connection_local(self): +- self.half_close_connection_local() ++ close_task = self.half_close_connection_local() + + with self.assertRaises(ConnectionClosed): + self.loop.run_until_complete(self.protocol.ping()) ++ + self.assertNoFrameSent() + ++ self.loop.run_until_complete(close_task) # cleanup ++ + def test_ping_on_closing_connection_remote(self): + self.half_close_connection_remote() + + with self.assertRaises(ConnectionClosed): + self.loop.run_until_complete(self.protocol.ping()) +- self.assertOneFrameSent(True, OP_CLOSE, serialize_close(1000, 'close')) ++ ++ self.assertNoFrameSent() + + def test_ping_on_closed_connection(self): + self.close_connection() + + with self.assertRaises(ConnectionClosed): + self.loop.run_until_complete(self.protocol.ping()) ++ + self.assertNoFrameSent() + + # Test the pong coroutine. +@@ -506,24 +545,29 @@ def test_pong_type_error(self): + self.assertNoFrameSent() + + def test_pong_on_closing_connection_local(self): +- self.half_close_connection_local() ++ close_task = self.half_close_connection_local() + + with self.assertRaises(ConnectionClosed): + self.loop.run_until_complete(self.protocol.pong()) ++ + self.assertNoFrameSent() + ++ self.loop.run_until_complete(close_task) # cleanup ++ + def test_pong_on_closing_connection_remote(self): + self.half_close_connection_remote() + + with self.assertRaises(ConnectionClosed): + self.loop.run_until_complete(self.protocol.pong()) +- self.assertOneFrameSent(True, OP_CLOSE, serialize_close(1000, 'close')) ++ ++ self.assertNoFrameSent() + + def test_pong_on_closed_connection(self): + self.close_connection() + + with self.assertRaises(ConnectionClosed): + self.loop.run_until_complete(self.protocol.pong()) ++ + self.assertNoFrameSent() + + # Test the protocol's logic for acknowledging pings with pongs.