From patchwork Tue Sep 22 16:11:47 2015 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Damien Lespiau X-Patchwork-Id: 521177 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [103.22.144.68]) (using TLSv1.2 with cipher AECDH-AES256-SHA (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 0866B1401AD for ; Wed, 23 Sep 2015 02:13:23 +1000 (AEST) Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id C4D961A01C0 for ; Wed, 23 Sep 2015 02:13:22 +1000 (AEST) X-Original-To: patchwork@lists.ozlabs.org Delivered-To: patchwork@lists.ozlabs.org Received: from mga01.intel.com (mga01.intel.com [192.55.52.88]) by lists.ozlabs.org (Postfix) with ESMTP id 988BC1A0024 for ; Wed, 23 Sep 2015 02:12:26 +1000 (AEST) Received: from fmsmga001.fm.intel.com ([10.253.24.23]) by fmsmga101.fm.intel.com with ESMTP; 22 Sep 2015 09:12:03 -0700 X-ExtLoop1: 1 X-IronPort-AV: E=Sophos;i="5.17,573,1437462000"; d="scan'208";a="794882580" Received: from pjanik-mobl1.ger.corp.intel.com (HELO strange.ger.corp.intel.com) ([10.252.32.80]) by fmsmga001.fm.intel.com with ESMTP; 22 Sep 2015 09:12:00 -0700 From: Damien Lespiau To: patchwork@lists.ozlabs.org Subject: [PATCH 4/6] tests: Add a couple of Selenium tests Date: Tue, 22 Sep 2015 17:11:47 +0100 Message-Id: <1442938309-3195-5-git-send-email-damien.lespiau@intel.com> X-Mailer: git-send-email 2.1.0 In-Reply-To: <1442938309-3195-1-git-send-email-damien.lespiau@intel.com> References: <1442938309-3195-1-git-send-email-damien.lespiau@intel.com> X-BeenThere: patchwork@lists.ozlabs.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: Patchwork development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: patchwork-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Patchwork" While developing the new series UI, several bug crept in but weren't discovered until later. All because we don't have in-browser tests to go along the lower level tests we already have. In particular, behaviours that need javascript to run cannot be tested outside of a full environment with the pages being served to an actual browser. This commit introduces selenium to the test suite and starts with 2 simple tests to give a taste of what it looks like. test_default_focus: make sure we do focus the username field on the login page test_login: shows how to chain actions to test the full login phase. This is quite similar the lower level test, except it also checks we display the username once logged in. v2: Use LiveServerTestCase for django pre-1.7 v3: Propagate the DISPLAY environment variable to have an X display specified for the browser v4: Log execution of the chrome driver, useful for debugging v5: Rebase on top of upstream Signed-off-by: Damien Lespiau --- .gitignore | 4 + docs/requirements-dev.txt | 1 + patchwork/tests/browser.py | 160 +++++++++++++++++++++++++++++++++++ patchwork/tests/test_user_browser.py | 38 +++++++++ tox.ini | 2 +- 5 files changed, 204 insertions(+), 1 deletion(-) create mode 100644 patchwork/tests/browser.py create mode 100644 patchwork/tests/test_user_browser.py diff --git a/.gitignore b/.gitignore index b16c5e2..52707be 100644 --- a/.gitignore +++ b/.gitignore @@ -28,3 +28,7 @@ cscope.* *.orig *.rej + +# test artifacts +/selenium.log +/selenium_screenshots diff --git a/docs/requirements-dev.txt b/docs/requirements-dev.txt index eee6463..12b8bef 100644 --- a/docs/requirements-dev.txt +++ b/docs/requirements-dev.txt @@ -1 +1,2 @@ -r requirements-base.txt +selenium diff --git a/patchwork/tests/browser.py b/patchwork/tests/browser.py new file mode 100644 index 0000000..9a6c4de --- /dev/null +++ b/patchwork/tests/browser.py @@ -0,0 +1,160 @@ +# Patchwork - automated patch tracking system +# Copyright (C) 2015 Intel Corporation +# +# This file is part of the Patchwork package. +# +# Patchwork is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# Patchwork is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with Patchwork; if not, write to the Free Software +# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + +import errno +import os +import time + +try: # django 1.7+ + from django.contrib.staticfiles.testing import StaticLiveServerTestCase +except: + from django.test import LiveServerTestCase as StaticLiveServerTestCase +from selenium import webdriver +from selenium.webdriver.support.ui import WebDriverWait +from selenium.common.exceptions import NoSuchElementException, \ + StaleElementReferenceException, TimeoutException + +class Wait(WebDriverWait): + """Subclass of WebDriverWait with predetermined timeout and poll + frequency. Also deals with a wider variety of exceptions.""" + _TIMEOUT = 10 + _POLL_FREQUENCY = 0.5 + + def __init__(self, driver): + super(Wait, self).__init__(driver, self._TIMEOUT, self._POLL_FREQUENCY) + + def until(self, method, message=''): + """Calls the method provided with the driver as an argument until the + return value is not False.""" + + end_time = time.time() + self._timeout + while True: + try: + value = method(self._driver) + if value: + return value + except NoSuchElementException: + pass + except StaleElementReferenceException: + pass + + time.sleep(self._poll) + if(time.time() > end_time): + break + + raise TimeoutException(message) + + def until_not(self, method, message=''): + """Calls the method provided with the driver as an argument until the + return value is False.""" + + end_time = time.time() + self._timeout + while(True): + try: + value = method(self._driver) + if not value: + return value + except NoSuchElementException: + return True + except StaleElementReferenceException: + pass + + time.sleep(self._poll) + if(time.time() > end_time): + break + + raise TimeoutException(message) + +def mkdir(path): + try: + os.makedirs(path) + except OSError as error: # Python >2.5 + if error.errno == errno.EEXIST and os.path.isdir(path): + pass + else: + raise + +class SeleniumTestCase(StaticLiveServerTestCase): + _SCREENSHOT_DIR = os.path.dirname(__file__) + '/../../selenium_screenshots' + + def setUp(self): + super(SeleniumTestCase, self).setUp() + + self.browser = os.getenv('SELENIUM_BROWSER', 'chrome') + if self.browser == 'firefox': + self.selenium = webdriver.Firefox() + if self.browser == 'chrome': + self.selenium = webdriver.Chrome( + service_args=["--verbose", "--log-path=selenium.log"] + ) + + mkdir(self._SCREENSHOT_DIR) + self._screenshot_number = 1 + + def tearDown(self): + self.selenium.quit() + super(SeleniumTestCase, self).tearDown() + + def screenshot(self): + name = "%s_%d.png" % (self._testMethodName, self._screenshot_number) + path = os.path.join(self._SCREENSHOT_DIR, name) + self.selenium.get_screenshot_as_file(path) + self._screenshot_number += 1 + + def get(self, relative_url): + self.selenium.get('%s%s' % (self.live_server_url, relative_url)) + self.screenshot() + + def find(self, selector): + return self.selenium.find_element_by_css_selector(selector) + + def focused_element(self): + return self.selenium.switch_to.active_element + + def wait_until_present(self, name): + is_present = lambda driver: driver.find_element_by_name(name) + msg = "An element named '%s' should be on the page" % name + element = Wait(self.selenium).until(is_present, msg) + self.screenshot() + return element + + def wait_until_visible(self, selector): + is_visible = lambda driver: self.find(selector).is_displayed() + msg = "The element matching '%s' should be visible" % selector + Wait(self.selenium).until(is_visible, msg) + self.screenshot() + return self.find(selector) + + def wait_until_focused(self, selector): + is_focused = \ + lambda driver: self.find(selector) == self.focused_element() + msg = "The element matching '%s' should be focused" % selector + Wait(self.selenium).until(is_focused, msg) + self.screenshot() + return self.find(selector) + + def enter_text(self, name, value): + field = self.wait_until_present(name) + field.send_keys(value) + return field + + def click(self, selector): + element = self.wait_until_visible(selector) + element.click() + return element diff --git a/patchwork/tests/test_user_browser.py b/patchwork/tests/test_user_browser.py new file mode 100644 index 0000000..2b9ed2e --- /dev/null +++ b/patchwork/tests/test_user_browser.py @@ -0,0 +1,38 @@ +# Patchwork - automated patch tracking system +# Copyright (C) 2015 Intel Corporation +# +# This file is part of the Patchwork package. +# +# Patchwork is free software; you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation; either version 2 of the License, or +# (at your option) any later version. +# +# Patchwork is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with Patchwork; if not, write to the Free Software +# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA + +from patchwork.tests.browser import SeleniumTestCase +from patchwork.tests.test_user import TestUser + +class LoginTestCase(SeleniumTestCase): + def setUp(self): + super(LoginTestCase, self).setUp() + self.user = TestUser() + + def test_default_focus(self): + self.get('/user/login/') + self.wait_until_focused('#id_username') + + def test_login(self): + self.get('/user/login/') + self.enter_text('username', self.user.username) + self.enter_text('password', self.user.password) + self.click('input[value="Login"]') + dropdown = self.wait_until_visible('a.dropdown-toggle strong') + self.assertEquals(dropdown.text, 'testuser') diff --git a/tox.ini b/tox.ini index 29c2d52..c04825e 100644 --- a/tox.ini +++ b/tox.ini @@ -17,7 +17,7 @@ commands = {toxinidir}/manage.py test --noinput '{posargs:patchwork}' passenv = http_proxy HTTP_PROXY https_proxy HTTPS_PROXY no_proxy NO_PROXY - PW_TEST_DB_USER PW_TEST_DB_PASS + PW_TEST_DB_USER PW_TEST_DB_PASS DISPLAY [testenv:pep8] basepython = python2.7