From patchwork Wed Mar 17 22:37:00 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Patchwork-Submitter: Valerio De Benedetto X-Patchwork-Id: 1455029 Return-Path: X-Original-To: incoming-buildroot@patchwork.ozlabs.org Delivered-To: patchwork-incoming-buildroot@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=busybox.net (client-ip=140.211.166.136; helo=smtp3.osuosl.org; envelope-from=buildroot-bounces@busybox.net; receiver=) Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (1024-bit key; unprotected) header.d=debevv.com header.i=@debevv.com header.a=rsa-sha256 header.s=protonmail header.b=dw1ocQ8R; dkim-atps=neutral Received: from smtp3.osuosl.org (smtp3.osuosl.org [140.211.166.136]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits) server-digest SHA256) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4F14m00Dxfz9sSC for ; Thu, 18 Mar 2021 09:37:20 +1100 (AEDT) Received: from localhost (localhost [127.0.0.1]) by smtp3.osuosl.org (Postfix) with ESMTP id 15D0A605CB; Wed, 17 Mar 2021 22:37:18 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Received: from smtp3.osuosl.org ([127.0.0.1]) by localhost (smtp3.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id cG8QwHIbCa0L; Wed, 17 Mar 2021 22:37:17 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by smtp3.osuosl.org (Postfix) with ESMTP id 276B66065A; Wed, 17 Mar 2021 22:37:16 +0000 (UTC) X-Original-To: buildroot@lists.busybox.net Delivered-To: buildroot@osuosl.org Received: from smtp2.osuosl.org (smtp2.osuosl.org [140.211.166.133]) by ash.osuosl.org (Postfix) with ESMTP id D8CE51BF21A for ; Wed, 17 Mar 2021 22:37:14 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by smtp2.osuosl.org (Postfix) with ESMTP id C7DD543096 for ; Wed, 17 Mar 2021 22:37:14 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org Authentication-Results: smtp2.osuosl.org (amavisd-new); dkim=pass (1024-bit key) header.d=debevv.com Received: from smtp2.osuosl.org ([127.0.0.1]) by localhost (smtp2.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id zB7Wpzj9ssF7 for ; Wed, 17 Mar 2021 22:37:13 +0000 (UTC) X-Greylist: from auto-whitelisted by SQLgrey-1.8.0 Received: from mail-40136.protonmail.ch (mail-40136.protonmail.ch [185.70.40.136]) by smtp2.osuosl.org (Postfix) with ESMTPS id B7CD14308E for ; Wed, 17 Mar 2021 22:37:12 +0000 (UTC) Date: Wed, 17 Mar 2021 22:37:00 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=debevv.com; s=protonmail; t=1616020628; bh=fp9t9+pEFqqcfwvU/rwZ27j5zzJZoUTA7/SVhT3CcYY=; h=Date:To:From:Cc:Reply-To:Subject:From; b=dw1ocQ8R0y+nCd0iKvMLj9+DgcENUDT94mIn+CglrnnhwWAlLkvJlWOMFtRrGlgJ+ w9aKiD8vrTRuSpy9zj8yg452HWQKN2FAgRuc2FtjvexCWSHmW1dEFhdpufDG1iGU4u FZqqaAHB8OrCuE+IDZlkT75VJ1bmnIxe0ykKD2F0= To: buildroot@buildroot.org From: Valerio De Benedetto Message-ID: <451fd672-7cde-127d-db41-df81eafcdbc5@debevv.com> MIME-Version: 1.0 Subject: [Buildroot] [PATCH 1/1] package/python-matplotlib: fix "Couldn't close file" runtime_error in pyplot import X-BeenThere: buildroot@busybox.net X-Mailman-Version: 2.1.29 Precedence: list List-Id: Discussion and development of buildroot List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Reply-To: Valerio De Benedetto Cc: jugurtha.belkalem@smile.fr, thomas.petazzoni@bootlin.com, asafka7@gmail.com Errors-To: buildroot-bounces@busybox.net Sender: "buildroot" This backports the PR from https://github.com/matplotlib/matplotlib/pull/15104 to matplotlib 3.0.3. Fixes the runtime_error thrown when executing "import matplotlib.pyplot" in a python3 shell (same symptoms of https://github.com/matplotlib/matplotlib/issues/15410) Signed-off-by: Valerio De Benedetto ---  ...03-Simplify-file-handling-in-ft2font.patch | 261 ++++++++++++++++++  1 file changed, 261 insertions(+)  create mode 100644 package/python-matplotlib/0003-Simplify-file-handling-in-ft2font.patch ++ ++exit: ++    Py_XDECREF(data); + +     return 0; + } +@@ -539,7 +472,6 @@ + static void PyFT2Font_dealloc(PyFT2Font *self) + { +     delete self->x; +-    free(self->mem); +     Py_XDECREF(self->py_file); +     Py_XDECREF(self->fname); +     Py_TYPE(self)->tp_free((PyObject *)self); -- 2.25.1 diff --git a/package/python-matplotlib/0003-Simplify-file-handling-in-ft2font.patch b/package/python-matplotlib/0003-Simplify-file-handling-in-ft2font.patch new file mode 100644 index 0000000000..392d1195f5 --- /dev/null +++ b/package/python-matplotlib/0003-Simplify-file-handling-in-ft2font.patch @@ -0,0 +1,261 @@ +--- a/src/ft2font_wrapper.cpp    2019-03-01 04:33:38.000000000 +0100 ++++ b/src/ft2font_wrapper.cpp    2021-03-17 02:22:07.580939585 +0100 +@@ -1,6 +1,5 @@ + #include "mplutils.h" + #include "ft2font.h" +-#include "file_compat.h" + #include "py_converters.h" + #include "py_exceptions.h" + #include "numpy_cpp.h" +@@ -328,12 +327,7 @@ +     FT2Font *x; +     PyObject *fname; +     PyObject *py_file; +-    FILE *fp; +-    int close_file; +-    mpl_off_t offset; +     FT_StreamRec stream; +-    FT_Byte *mem; +-    size_t mem_size; +     Py_ssize_t shape[2]; +     Py_ssize_t strides[2]; +     Py_ssize_t suboffsets[2]; +@@ -344,118 +338,43 @@ +                                              unsigned char *buffer, +                                              unsigned long count) + { +- +-    PyFT2Font *def = (PyFT2Font *)stream->descriptor.pointer; +- +-    if (fseek(def->fp, offset, SEEK_SET) == -1) { +-        return 0; ++    PyObject *py_file = ((PyFT2Font *)stream->descriptor.pointer)->py_file; ++    PyObject *seek_result = NULL, *read_result = NULL; ++    Py_ssize_t n_read = 0; ++    if (!(seek_result = PyObject_CallMethod(py_file, "seek", "k", offset)) ++        || !(read_result = PyObject_CallMethod(py_file, "read", "k", count))) { ++        goto exit; ++    } ++    char *tmpbuf; ++    if (PyBytes_AsStringAndSize(read_result, &tmpbuf, &n_read) == -1) { ++        goto exit; +     } +- +-    if (count > 0) { +-        return fread(buffer, 1, count, def->fp); ++    memcpy(buffer, tmpbuf, n_read); ++exit: ++    Py_XDECREF(seek_result); ++    Py_XDECREF(read_result); ++    if (PyErr_Occurred()) { ++        PyErr_WriteUnraisable(py_file); ++        if (!count) { ++            return 1;  // Non-zero signals error, when count == 0. ++        } +     } +- +-    return 0; ++    return n_read; + } + + static void close_file_callback(FT_Stream stream) + { +-    PyFT2Font *def = (PyFT2Font *)stream->descriptor.pointer; +- +-    if (mpl_PyFile_DupClose(def->py_file, def->fp, def->offset)) { +-        throw std::runtime_error("Couldn't close file"); +-    } +- +-    if (def->close_file) { +-        mpl_PyFile_CloseFile(def->py_file); +-    } +- +-    Py_DECREF(def->py_file); +-    def->py_file = NULL; +-} +- +-static int convert_open_args(PyFT2Font *self, PyObject *py_file_arg, FT_Open_Args *open_args) +-{ +-    PyObject *py_file = NULL; +-    int close_file = 0; +-    FILE *fp; +-    PyObject *data = NULL; +-    char *data_ptr; +-    Py_ssize_t data_len; +-    long file_size; +-    FT_Byte *new_memory; +-    mpl_off_t offset = 0; +- +-    int result = 0; +- +-    memset((void *)open_args, 0, sizeof(FT_Open_Args)); +- +-    if (PyBytes_Check(py_file_arg) || PyUnicode_Check(py_file_arg)) { +-        if ((py_file = mpl_PyFile_OpenFile(py_file_arg, (char *)"rb")) == NULL) { +-            goto exit; +-        } +-        close_file = 1; +-    } else { +-        Py_INCREF(py_file_arg); +-        py_file = py_file_arg; +-    } +- +-    if ((fp = mpl_PyFile_Dup(py_file, (char *)"rb", &offset))) { +-        Py_INCREF(py_file); +-        self->py_file = py_file; +-        self->close_file = close_file; +-        self->fp = fp; +-        self->offset = offset; +-        fseek(fp, 0, SEEK_END); +-        file_size = ftell(fp); +-        fseek(fp, 0, SEEK_SET); +- +-        self->stream.base = NULL; +-        self->stream.size = (unsigned long)file_size; +-        self->stream.pos = 0; +-        self->stream.descriptor.pointer = self; +-        self->stream.read = &read_from_file_callback; +-        self->stream.close = &close_file_callback; +- +-        open_args->flags = FT_OPEN_STREAM; +-        open_args->stream = &self->stream; +-    } else { +-        if (PyObject_HasAttrString(py_file_arg, "read") && +-            (data = PyObject_CallMethod(py_file_arg, (char *)"read", (char *)""))) { +-            if (PyBytes_AsStringAndSize(data, &data_ptr, &data_len)) { +-                goto exit; +-            } +- +-            if (self->mem) { +-                free(self->mem); +-            } +-            self->mem = (FT_Byte *)malloc((self->mem_size + data_len) * sizeof(FT_Byte)); +-            if (self->mem == NULL) { +-                goto exit; +-            } +-            new_memory = self->mem + self->mem_size; +-            self->mem_size += data_len; +- +-            memcpy(new_memory, data_ptr, data_len); +-            open_args->flags = FT_OPEN_MEMORY; +-            open_args->memory_base = new_memory; +-            open_args->memory_size = data_len; +-            open_args->stream = NULL; +-        } else { +-            PyErr_SetString(PyExc_TypeError, +-                            "First argument must be a path or file object reading bytes"); +-            goto exit; +-        } ++    PyFT2Font *self = (PyFT2Font *)stream->descriptor.pointer; ++    PyObject *close_result = NULL; ++    if (!(close_result = PyObject_CallMethod(self->py_file, "close", ""))) { ++        goto exit; +     } +- +-    result = 1; +- + exit: +- +-    Py_XDECREF(py_file); +-    Py_XDECREF(data); +- +-    return result; ++    Py_XDECREF(close_result); ++    Py_CLEAR(self->py_file); ++    if (PyErr_Occurred()) { ++        PyErr_WriteUnraisable((PyObject*)self); ++    } + } + + static PyTypeObject PyFT2FontType; +@@ -467,12 +386,7 @@ +     self->x = NULL; +     self->fname = NULL; +     self->py_file = NULL; +-    self->fp = NULL; +-    self->close_file = 0; +-    self->offset = 0; +     memset(&self->stream, 0, sizeof(FT_StreamRec)); +-    self->mem = 0; +-    self->mem_size = 0; +     return (PyObject *)self; + } + +@@ -503,35 +417,54 @@ +     "  underline_thickness    vertical thickness of the underline\n" +     "  postscript_name        PostScript name of the font\n"; + +-static void PyFT2Font_fail(PyFT2Font *self) +-{ +-    free(self->mem); +-    self->mem = NULL; +-    Py_XDECREF(self->py_file); +-    self->py_file = NULL; +-} +- + static int PyFT2Font_init(PyFT2Font *self, PyObject *args, PyObject *kwds) + { +-    PyObject *fname; ++    PyObject *filename = NULL, *open = NULL, *data = NULL; +     FT_Open_Args open_args; +     long hinting_factor = 8; +     const char *names[] = { "filename", "hinting_factor", NULL }; + +     if (!PyArg_ParseTupleAndKeywords( +-             args, kwds, "O|l:FT2Font", (char **)names, &fname, &hinting_factor)) { ++             args, kwds, "O|l:FT2Font", (char **)names, &filename, &hinting_factor)) { +         return -1; +     } + +-    if (!convert_open_args(self, fname, &open_args)) { +-        return -1; ++    self->stream.base = NULL; ++    self->stream.size = 0x7fffffff;  // Unknown size. ++    self->stream.pos = 0; ++    self->stream.descriptor.pointer = self; ++    self->stream.read = &read_from_file_callback; ++    memset((void *)&open_args, 0, sizeof(FT_Open_Args)); ++    open_args.flags = FT_OPEN_STREAM; ++    open_args.stream = &self->stream; ++ ++    if (PyBytes_Check(filename) || PyUnicode_Check(filename)) { ++        if (!(open = PyDict_GetItemString(PyEval_GetBuiltins(), "open"))  // Borrowed reference. ++            || !(self->py_file = PyObject_CallFunction(open, "Os", filename, "rb"))) { ++            goto exit; ++        } ++        self->stream.close = &close_file_callback; ++    } else if (!PyObject_HasAttrString(filename, "read") ++               || !(data = PyObject_CallMethod(filename, "read", "i", 0)) ++               || !PyBytes_Check(data)) { ++        PyErr_SetString(PyExc_TypeError, ++                        "First argument must be a path or binary-mode file object"); ++        goto exit; ++    } else { ++        self->py_file = filename; ++        self->stream.close = NULL; ++        Py_INCREF(filename); +     } + +     CALL_CPP_FULL( +-        "FT2Font", (self->x = new FT2Font(open_args, hinting_factor)), PyFT2Font_fail(self), -1); ++        "FT2Font", (self->x = new FT2Font(open_args, hinting_factor)), ++        Py_CLEAR(self->py_file), -1); + +-    Py_INCREF(fname); +-    self->fname = fname; ++    Py_INCREF(filename); ++    self->fname = filename;