Improvements to python client example

Hi,

we are using the example python client in our test stations and have done
some improvements on the way:

- Start websocket client at the beginning of the update, so all messages
are shown, not only the messages that are received after finishing the
upload
- Call requests.post in executor to not block the asyncio event loop
- Use argparse to handle command line arguments
- Handle timeout in both the upload task and the websocket task
- Use flake8/black to lint and format the code
- Change file name to swupdate_client.py, so it can be included from other
python modules
- Add support for logging, so external programs can register their own
logger (in our case, we are logging directly to OpenHTF)
- Handle json parse error in the version info message
- Add Pipfile and README.md with documentation on how to install
dependencies and run the program

Best regards
Johannes Schrimpf
Blueye Robotics


From ef6a26f0527013ff2aed9cfcec39cb1b7e0ae396 Mon Sep 17 00:00:00 2001
From: Johannes Schrimpf <johannes.schrimpf@blueye.no>
Date: Tue, 26 Oct 2021 15:21:31 +0200
Subject: [PATCH 1/3] Rename swupdate-client.py to swupdate_client.py

Change file name to swupdate_client.py, so it can be included from other
python modules

Signed-off-by: Johannes Schrimpf <johannes.schrimpf@blueye.no>
---
 examples/client/{swupdate-client.py => swupdate_client.py} | 0
 1 file changed, 0 insertions(+), 0 deletions(-)
 rename examples/client/{swupdate-client.py => swupdate_client.py} (100%)

Hi Johannes,

On 26.10.21 15:54, Johannes Schrimpf wrote:
> Hi,
> 
> we are using the example python client in our test stations and have 
> done some improvements on the way:
> 
> - Start websocket client at the beginning of the update, so all messages 
> are shown, not only the messages that are received after finishing the 
> upload
> - Call requests.post <http://requests.post> in executor to not block the 
> asyncio event loop
> - Use argparse to handle command line arguments
> - Handle timeout in both the upload task and the websocket task
> - Use flake8/black to lint and format the code
> - Change file name to swupdate_client.py, so it can be included from 
> other python modules
> - Add support for logging, so external programs can register their own 
> logger (in our case, we are logging directly to OpenHTF)
> - Handle json parse error in the version info message
> - Add Pipfile and README.md with documentation on how to install 
> dependencies and run the program
> 

Thanks for this. Could you please these patches with git send-email 
instead of copy & paste, else your mailer will mess up the patches ?

just "git send-email --to swupdate@googlegroups.com 00*"

Thanks,
Stefano Babic

> Best regards
> Johannes Schrimpf
> Blueye Robotics
> 
> 
>  From ef6a26f0527013ff2aed9cfcec39cb1b7e0ae396 Mon Sep 17 00:00:00 2001
> From: Johannes Schrimpf <johannes.schrimpf@blueye.no 
> <mailto:johannes.schrimpf@blueye.no>>
> Date: Tue, 26 Oct 2021 15:21:31 +0200
> Subject: [PATCH 1/3] Rename swupdate-client.py to swupdate_client.py
> 
> Change file name to swupdate_client.py, so it can be included from other 
> python modules
> 
> Signed-off-by: Johannes Schrimpf <johannes.schrimpf@blueye.no 
> <mailto:johannes.schrimpf@blueye.no>>
> ---
>   examples/client/{swupdate-client.py => swupdate_client.py} | 0
>   1 file changed, 0 insertions(+), 0 deletions(-)
>   rename examples/client/{swupdate-client.py => swupdate_client.py} (100%)
> 
> diff --git a/examples/client/swupdate-client.py 
> b/examples/client/swupdate_client.py
> similarity index 100%
> rename from examples/client/swupdate-client.py
> rename to examples/client/swupdate_client.py
> -- 
> 2.30.1 (Apple Git-130)
> 
> 
> 
>  From 303c093b04d7d7b8ce6668c7b4a3c5165bd1b83a Mon Sep 17 00:00:00 2001
> From: Johannes Schrimpf <johannes.schrimpf@blueye.no 
> <mailto:johannes.schrimpf@blueye.no>>
> Date: Tue, 26 Oct 2021 15:22:26 +0200
> Subject: [PATCH 2/3] Improve example client
> 
> - Start websocket client at the beginning of the update, so all messages 
> are shown
> - Call requests.post <http://requests.post> in executor to not block the 
> asyncio event loop
> - Use argparse to handle command line arguments
> - Handle timeout in both the upload task and the websocket task
> - Use flake8/black to lint and format code
> - Add support for logging, so external programs can register their own 
> logger
> - Handle json parse error in version info message
> 
> Signed-off-by: Johannes Schrimpf <johannes.schrimpf@blueye.no 
> <mailto:johannes.schrimpf@blueye.no>>
> ---
>   examples/client/swupdate_client.py | 158 +++++++++++++++++++++--------
>   1 file changed, 113 insertions(+), 45 deletions(-)
> 
> diff --git a/examples/client/swupdate_client.py 
> b/examples/client/swupdate_client.py
> index ae31ec9..fc0c1d4 100755
> --- a/examples/client/swupdate_client.py
> +++ b/examples/client/swupdate_client.py
> @@ -6,68 +6,136 @@
> 
>   import asyncio
>   import json
> -import os
>   import requests
>   import websockets
> +import logging
> +import string
> +import argparse
>   import sys
> 
> 
>   class SWUpdater:
> -    "" " Python helper class for SWUpdate " ""
> -
> -    url_upload = 'http://{}:{}/upload'
> -    url_status = 'ws://{}:{}/ws'
> -
> -    def __init__ (self, path_image, host_name, port):
> -        self.__image = path_image
> -        self.__host_name = host_name
> -        self.__port = port
> -
> -
> -    async def wait_update_finished(self, timeout = 300):
> -        print ("Wait update finished")
> -        async def get_finish_messages ():
> -            async with 
> websockets.connect(self.url_status.format(self.__host_name, 
> self.__port)) as websocket:
> +    """Python helper class for SWUpdate"""
> +
> +    url_upload = "http://{}:{}/upload"
> +    url_status = "ws://{}:{}/ws"
> +
> +    def __init__(self, path_image, host_name, port=8080, logger=None):
> +        self._image = path_image
> +        self._host_name = host_name
> +        self._port = port
> +        if logger is not None:
> +            self._logger = logger
> +        else:
> +            logging.basicConfig(stream=sys.stdout, level=logging.INFO)
> +            self._logger = logging.getLogger("swupdate")
> +
> +    async def wait_update_finished(self):
> +        self._logger.info <http://logger.info>("Waiting for messages on 
> websocket connection")
> +        try:
> +            async with websockets.connect(
> +                self.url_status.format(self._host_name, self._port)
> +            ) as websocket:
>                   while True:
> -                    message = await websocket.recv()
> -                    data = json.loads(message)
> -
> -                    if data ["type"] != "message":
> +                    try:
> +                        message = await websocket.recv()
> +                        message = "".join(
> +                            filter(lambda x: x in 
> set(string.printable), message)
> +                        )
> +
> +                    except Exception as err:
> +                        self._logger.warning(err)
>                           continue
> 
> -                    print (data["text"])
> -                    if data ["text"] == "SWUPDATE successful !":
> -                        return
> +                    try:
> +                        data = json.loads(message)
> +                    except json.decoder.JSONDecodeError:
> +                        # As of 2021.04, the version info message 
> contains invalid json
> +                        self._logger.warning(f"json parse error: 
> {message}")
> +                        continue
> 
> -        await asyncio.wait_for(get_finish_messages(), timeout = timeout)
> +                    if data["type"] != "message":
> +                        continue
> 
> -    def update (self, timeout = 300):
> -        print ("Start uploading image...")
> -        print (self.url_upload.format(self.__host_name, self.__port))
> +                    self._logger.info <http://logger.info>(data["text"])
> +                    if "SWUPDATE successful" in data["text"]:
> +                        return True
> +                    if "Installation failed" in data["text"]:
> +                        return False
> +
> +        except Exception as err:
> +            self._logger.error(err)
> +            return False
> +
> +    def sync_upload(self, swu_file, timeout):
> +        return requests.post <http://requests.post>(
> +            self.url_upload.format(self._host_name, self._port),
> +            files={"file": swu_file},
> +            timeout=timeout,
> +        )
> +
> +    async def upload(self, timeout):
> +        self._logger.info <http://logger.info>("Start uploading image...")
>           try:
> -            response = requests.post 
> <http://requests.post>(self.url_upload.format(self.__host_name, 
> self.__port), files = { 'file':open (self.__image, 'rb') })
> +            with open(self._image, "rb") as swu_file:
> +                loop = asyncio.get_event_loop()
> +                response = await loop.run_in_executor(
> +                    None, self.sync_upload, swu_file, timeout
> +                )
> 
>               if response.status_code != 200:
> -                raise Exception ("Cannot upload software image: {}". 
>   format (response.status_code))
> -
> -            print ("Software image uploaded successfully. Wait for 
> installation to be finished...\n")
> -            asyncio.sleep(10)
> -           
>   asyncio.get_event_loop().run_until_complete(self.wait_update_finished(timeout = timeout))
> -
> +                self._logger.error(
> +                    "Cannot upload software image: 
> {}".format(response.status_code)
> +                )
> +                return False
> +
> +            self._logger.info <http://logger.info>(
> +                "Software image uploaded successfully."
> +                "Wait for installation to be finished..."
> +            )
> +            return True
>           except ValueError:
> -            print("No connection to host, exit")
> +            self._logger.info <http://logger.info>("No connection to 
> host, exit")
> +        except FileNotFoundError:
> +            self._logger.info <http://logger.info>("swu file not found")
> +        except requests.exceptions.ConnectionError as e:
> +            self._logger.info <http://logger.info>("Connection 
> Error:\n%s" % str(e))
> +        return False
> +
> +    async def start_tasks(self, timeout):
> +        ws_task = asyncio.create_task(self.wait_update_finished())
> +        upload_task = asyncio.create_task(self.upload(timeout))
> +
> +        if not await upload_task:
> +            self._logger.info <http://logger.info>("Cancelling 
> websocket task")
> +            ws_task.cancel()
> +            return False
> 
> +        try:
> +            result = await asyncio.wait_for(ws_task, timeout=timeout)
> +        except asyncio.TimeoutError:
> +            self._logger.info <http://logger.info>("timeout!")
> +            return False
> 
> -if __name__ == "__main__":
> -    sys.path.append (os.getcwd ())
> +        return result
> 
> -    if len (sys.argv) == 3:
> -        port = "8080"
> -    elif len (sys.argv) == 4:
> -        port = sys.argv[3]
> -    else:
> -       print ("Usage: swupdate.py <path to image> <hostname> [port]")
> -       exit (1)
> +    def update(self, timeout=300):
> +        return asyncio.run(self.start_tasks(timeout))
> 
> 
> -    SWUpdater (sys.argv[1], sys.argv[2], port).update ()
> +if __name__ == "__main__":
> +    parser = argparse.ArgumentParser()
> +    parser.add_argument("swu_file", help="Path to swu image")
> +    parser.add_argument("host_name", help="Host name")
> +    parser.add_argument("port", help="Port", type=int, default=8080, 
> nargs="?")
> +    parser.add_argument(
> +        "--timeout",
> +        help="Timeout for the whole swupdate process",
> +        type=int,
> +        default=300,
> +        nargs="?",
> +    )
> +
> +    args = parser.parse_args()
> +    updater = SWUpdater(args.swu_file, args.host_name, args.port)
> +    updater.update(timeout=args.timeout)
> -- 
> 2.30.1 (Apple Git-130)
> 
> 
>  From b0401b4d85d9f6ce6cbca91653f133872cb8cd72 Mon Sep 17 00:00:00 2001
> From: Johannes Schrimpf <johannes.schrimpf@blueye.no 
> <mailto:johannes.schrimpf@blueye.no>>
> Date: Tue, 26 Oct 2021 15:26:53 +0200
> Subject: [PATCH 3/3] Add Pipfile and README.md
> 
> Signed-off-by: Johannes Schrimpf <johannes.schrimpf@blueye.no 
> <mailto:johannes.schrimpf@blueye.no>>
> ---
>   examples/client/Pipfile   | 13 +++++++++++
>   examples/client/README.md | 47 +++++++++++++++++++++++++++++++++++++++
>   2 files changed, 60 insertions(+)
>   create mode 100644 examples/client/Pipfile
>   create mode 100644 examples/client/README.md
> 
> diff --git a/examples/client/Pipfile b/examples/client/Pipfile
> new file mode 100644
> index 0000000..a237601
> --- /dev/null
> +++ b/examples/client/Pipfile
> @@ -0,0 +1,13 @@
> +[[source]]
> +url = "https://pypi.org/simple <https://pypi.org/simple>"
> +verify_ssl = true
> +name = "pypi"
> +
> +[packages]
> +websockets = "*"
> +requests = "*"
> +
> +[dev-packages]
> +
> +[requires]
> +python_version = "3.9"
> diff --git a/examples/client/README.md b/examples/client/README.md
> new file mode 100644
> index 0000000..b6e5d32
> --- /dev/null
> +++ b/examples/client/README.md
> @@ -0,0 +1,47 @@
> +## Install dependencies
> +
> +### apt
> +```
> +sudo apt update
> +sudo apt install python3-websockets python3-requests
> +```
> +
> +### pip
> +```
> +pip install websockets requests
> +```
> +
> +### pipenv
> +```
> +pipenv install
> +```
> +
> +## Usage
> +
> +### apt/pip
> +```
> +./swupdate_client.py <path-to-swu> <host_name> [port]
> +```
> +
> +### pipenv
> +```
> +pipenv run ./swupdate_client.py <path-to-swu> <host_name> [port]
> +```
> +
> +
> +## Development
> +### Import from another python program
> +```
> +from swupdate_client import SWUpdater
> +
> +updater = SWUpdater("path-to-swu", "host-name")
> +if updater.update():
> +    print("Update successful!")
> +else:
> +    print("Update failed!")
> +```
> +
> +### Formatting
> +```
> +black swupdate_client.py
> +```
> \ No newline at end of file
> -- 
> 2.30.1 (Apple Git-130)
> 
> 
> -- 
> You received this message because you are subscribed to the Google 
> Groups "swupdate" group.
> To unsubscribe from this group and stop receiving emails from it, send 
> an email to swupdate+unsubscribe@googlegroups.com 
> <mailto:swupdate+unsubscribe@googlegroups.com>.
> To view this discussion on the web visit 
> https://groups.google.com/d/msgid/swupdate/CAJ_nv-B47WT6HDWZa%2BSREhcTfVmzDbsgFTXCfgT9aubg_2MT%2Bg%40mail.gmail.com 
> <https://groups.google.com/d/msgid/swupdate/CAJ_nv-B47WT6HDWZa%2BSREhcTfVmzDbsgFTXCfgT9aubg_2MT%2Bg%40mail.gmail.com?utm_medium=email&utm_source=footer>.