From patchwork Sat Feb 27 13:08:35 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Heinrich Schuchardt X-Patchwork-Id: 1445171 X-Patchwork-Delegate: xypron.glpk@gmx.de Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=2a01:238:438b:c500:173d:9f52:ddab:ee01; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; secure) header.d=gmx.net header.i=@gmx.net header.a=rsa-sha256 header.s=badeba3b8450 header.b=N9rCHooW; dkim-atps=neutral Received: from phobos.denx.de (phobos.denx.de [IPv6:2a01:238:438b:c500:173d:9f52:ddab:ee01]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4Dnn152lYvz9sBJ for ; Sun, 28 Feb 2021 00:09:29 +1100 (AEDT) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id C84D382036; Sat, 27 Feb 2021 14:09:24 +0100 (CET) Authentication-Results: phobos.denx.de; dmarc=fail (p=none dis=none) header.from=gmx.de Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; secure) header.d=gmx.net header.i=@gmx.net header.b="N9rCHooW"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id 6171081FC1; Sat, 27 Feb 2021 14:09:10 +0100 (CET) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,FREEMAIL_FROM,RCVD_IN_MSPIKE_H2,SPF_HELO_NONE autolearn=ham autolearn_force=no version=3.4.2 Received: from mout.gmx.net (mout.gmx.net [212.227.17.20]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id 8533C81F66 for ; Sat, 27 Feb 2021 14:09:05 +0100 (CET) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=gmx.de Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=xypron.glpk@gmx.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gmx.net; s=badeba3b8450; t=1614431344; bh=dTdjIYKvYpzWCec2oKsw8/NlFGUADUrJVbdp+iPCcR4=; h=X-UI-Sender-Class:From:To:Cc:Subject:Date:In-Reply-To:References; b=N9rCHooWJm+ARtwEsVMeBLQ8851D4ewTeFT/jqTCowuiymaD5CXn23z7mjaY6k8t5 hrCjZW3zKDZbIJoY+jxpj4ylOtghAKby+lOPukxgNpYDrnaIhAhVgFJCB1C/BAHcfz UYIjrvas2pFOqS+yFO0SV3O9+ALFxTVM+PZVnHNU= X-UI-Sender-Class: 01bb95c1-4bf8-414a-932a-4f6e2808ef9c Received: from LT02.fritz.box ([62.143.246.89]) by mail.gmx.net (mrgmx105 [212.227.17.174]) with ESMTPSA (Nemesis) id 1Mw9UK-1m7wwO1T5Y-00s9T3; Sat, 27 Feb 2021 14:09:04 +0100 From: Heinrich Schuchardt To: Alexander Graf , Anatolij Gustschin Cc: u-boot@lists.denx.de, Heinrich Schuchardt Subject: [PATCH 1/6] efi_loader: move codepage 437 table Date: Sat, 27 Feb 2021 14:08:35 +0100 Message-Id: <20210227130840.166193-2-xypron.glpk@gmx.de> X-Mailer: git-send-email 2.30.0 In-Reply-To: <20210227130840.166193-1-xypron.glpk@gmx.de> References: <20210227130840.166193-1-xypron.glpk@gmx.de> MIME-Version: 1.0 X-Provags-ID: V03:K1:ozWXfqf9RZmgZ7TjLOKudONTAYjE9gbwWEMRLLgPV4wKVjtpBpV 3HEoiGWW9ojZzZU9OdEVKnjAREZWmcRkN90ZJvD4E8UCyuB2MUhpamjHPSDOb2ujdEtLCiD Lu111owT0cgDZzOQMCKLMNoL88reg851LiPLTXQaXIHEnldBP2pY/7vANuhYuO+RUbiYpVn dM+Say+PaQfHIiVLJYU1A== X-UI-Out-Filterresults: notjunk:1;V03:K0:5MBeyopmow8=:BTGrHOfhEgLeejpI/MGLEu c0+/FsdYCLGssvuCJp8IP20wkb3ZnitZehscYvnZhk8EXP25QWCxaY8kclSjLS6Epu2gyTB4+ C4hK/l0F6/Kj69qGuBf/0x7lrpEZdSHnk0onz1SUXGJb+15WYTPzaxhktjobBJmqiKVFRvveL hY8ME58cZ0Q5Y3p7PdhU5TZ10k86Ia0pWLLOhKcfAaIJW0QOUIBZP8XM3/b58Uq4IC9L34hT6 0FEITAoDJfuR783FoILp6oAXLP9pwrjvkvc9q/KGT+jFFYkQgKaA+MmZ00r+HFCKnoRC7/O4G XKeeT5V66wU6ZuSPbWM7S99xgDvgqfAfOdcwRj5rNb8BWyy+gQAOLIQAmetQER5YhDyMN/Dqs wwCT3k+x8WaApyLCblpXdwpyMMmvTXX0QucR3HG22KzcnSi26AgEBxQShMm6cgvi81Sgr+bzz l8lVZ9zeWXPm6fDVwkDGQqmY1NjvxWg1vHdXuOGeR46kd2ewihEm6bC3+h2shGSD5HsZoRvl/ 92sMiQEE+H+NeKQngR3ydu0pQrgi6hL3Pt9cdBouHrxsDHEo8U026SCqdmbxlq5dze/GfYC0M VZyhRHo2bj1Iq3z3U7BXkCc4+SYuBnFJ0eyaJt5YwFbgxzzEVKDVEfaUojAD7KMgK36dnFGU/ AgKCnaPA9/YyIWKM+1gix+eAEEm9PWQq83LrNHnHsq/rNfQSD38Q66kO7Y2dJ6s3g43kpKxYI UcA+E8UAxw7ZaMxQwI1MWYLmr9DzH/TZpsxNXPzVviH0QWI0hgfoqaDewY7On82N7W2Gflwuj dLi+C3fuss9lPi37krP3xiJ3Z6Ypz0mXmlDQX7uf4B87pOlIVgV2p0h4QyPh9nECCDHUV10Zb 0RbgFxKy1qgYGPVDf3Eg== X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.34 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.102.4 at phobos.denx.de X-Virus-Status: Clean Move the Unicode to codepage 437 table to charset.c Signed-off-by: Heinrich Schuchardt --- include/charset.h | 5 +++++ lib/charset.c | 6 ++++++ lib/efi_loader/efi_unicode_collation.c | 2 +- 3 files changed, 12 insertions(+), 1 deletion(-) -- 2.30.0 diff --git a/include/charset.h b/include/charset.h index cc650a2ce7..64ba91f791 100644 --- a/include/charset.h +++ b/include/charset.h @@ -13,6 +13,11 @@ #define MAX_UTF8_PER_UTF16 3 +/** + * codepage_437 - Unicode to codepage 437 translation table + */ +extern const u16 codepage_437[128]; + /** * console_read_unicode() - read Unicode code point from console * diff --git a/lib/charset.c b/lib/charset.c index 2177014ee1..814847d165 100644 --- a/lib/charset.c +++ b/lib/charset.c @@ -8,9 +8,15 @@ #include #include #include +#include #include #include +/** + * codepage_437 - Unicode to codepage 437 translation table + */ +const u16 codepage_437[128] = CP437; + static struct capitalization_table capitalization_table[] = #ifdef CONFIG_EFI_UNICODE_CAPITALIZATION UNICODE_CAPITALIZATION_TABLE; diff --git a/lib/efi_loader/efi_unicode_collation.c b/lib/efi_loader/efi_unicode_collation.c index f6c875bc33..bf5314c4ff 100644 --- a/lib/efi_loader/efi_unicode_collation.c +++ b/lib/efi_loader/efi_unicode_collation.c @@ -23,7 +23,7 @@ static const char illegal[] = "+,<=>:;\"/\\|?*[]\x7f"; static const u16 codepage[] = CP1250; #else /* Unicode code points for code page 437 characters 0x80 - 0xff */ -static const u16 codepage[] = CP437; +static const u16 *codepage = codepage_437; #endif /* GUID of the EFI_UNICODE_COLLATION_PROTOCOL2 */ From patchwork Sat Feb 27 13:08:36 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Heinrich Schuchardt X-Patchwork-Id: 1445173 X-Patchwork-Delegate: xypron.glpk@gmx.de Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=2a01:238:438b:c500:173d:9f52:ddab:ee01; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; secure) header.d=gmx.net header.i=@gmx.net header.a=rsa-sha256 header.s=badeba3b8450 header.b=eGXyIdzO; dkim-atps=neutral Received: from phobos.denx.de (phobos.denx.de [IPv6:2a01:238:438b:c500:173d:9f52:ddab:ee01]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4Dnn1W6dfPz9sBJ for ; Sun, 28 Feb 2021 00:09:51 +1100 (AEDT) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id 2568B82666; Sat, 27 Feb 2021 14:09:40 +0100 (CET) Authentication-Results: phobos.denx.de; dmarc=fail (p=none dis=none) header.from=gmx.de Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; secure) header.d=gmx.net header.i=@gmx.net header.b="eGXyIdzO"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id A0EC781FDF; Sat, 27 Feb 2021 14:09:16 +0100 (CET) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,FREEMAIL_FROM,RCVD_IN_MSPIKE_H2,SPF_HELO_NONE autolearn=ham autolearn_force=no version=3.4.2 Received: from mout.gmx.net (mout.gmx.net [212.227.17.21]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id DF89B81F9C for ; Sat, 27 Feb 2021 14:09:05 +0100 (CET) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=gmx.de Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=xypron.glpk@gmx.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gmx.net; s=badeba3b8450; t=1614431344; bh=UwfCqCCSGhipctwe33J/rZWgsmhd2tcZZLPCeGiOoJU=; h=X-UI-Sender-Class:From:To:Cc:Subject:Date:In-Reply-To:References; b=eGXyIdzOAe265mJNa4HmmWNz85/N1FtVBpL6aaJyI0h9LnKG/ShSqkrzVOyG1XXAF RsH1I4K48oVal+BHiIYPcIT5En2sCxbgoHEdKAgC3gAJJPRqObkp5GBKIgOYqP7YAE 3QQuQCPg/IELkxihoyE9LxsUi7weWbS7l36q+VTg= X-UI-Sender-Class: 01bb95c1-4bf8-414a-932a-4f6e2808ef9c Received: from LT02.fritz.box ([62.143.246.89]) by mail.gmx.net (mrgmx105 [212.227.17.174]) with ESMTPSA (Nemesis) id 1N8ofO-1luFoU2QuE-015shb; Sat, 27 Feb 2021 14:09:04 +0100 From: Heinrich Schuchardt To: Alexander Graf , Anatolij Gustschin Cc: u-boot@lists.denx.de, Heinrich Schuchardt Subject: [PATCH 2/6] efi_loader: carve out utf_to_cp() Date: Sat, 27 Feb 2021 14:08:36 +0100 Message-Id: <20210227130840.166193-3-xypron.glpk@gmx.de> X-Mailer: git-send-email 2.30.0 In-Reply-To: <20210227130840.166193-1-xypron.glpk@gmx.de> References: <20210227130840.166193-1-xypron.glpk@gmx.de> MIME-Version: 1.0 X-Provags-ID: V03:K1:56UGaXPGysadcC+LPHOwM9//WK9F9CkxfsXEQKWa22wge9y45bR oLYT9yECTZ2NDdwukf/BzDwVtlypeytXVzKXM+cfoLzVwuN3vITKzHsSvHzW+F5k2YAiZMu ZiNszRtZIUG0uIboWDe2/zbDg9Y7MKgyixf9g8ziiY+YV8psp6DLyem9/GATIEEWh6umSk+ vIAQKGaJpGKQx/6xY28fA== X-UI-Out-Filterresults: notjunk:1;V03:K0:QoEZhRN7U5o=:P6R438BRy2LAgs6nXwnlBu tVyIWgaCQPMUUgCw2+tDlbrmrzF4BCAzFdC7s9cjtXAIFCJm4hxQU2Yu/EW6Gw1/Ssx32gjJH sZsgtyHa8wS3fj1btJSLTcts2NvP/pWBOPYsrwOC4FP8O1CelYfAJb3e2RO88/GtLtXo7VQqE hNxe+UDGL7VdRRhkW1qwOGN2aDxX9g0Exe+f+BcpMsMLZghAOrrg0KNrHmGvJ4J1ddb1arycU Bg4wYge3SfOnUgZkyup9RcRkonbR4yLXGOOQKJ3uHpeDQGXivhqg07MfJARK0sbhe9f2bAbdD 3Mou7qCObDfimFHhwsluoJ2le6dlzIdE3YVTgELL7sssLtCq92pNjDSA9Bo9Jmt2skLuZrm7z /K5AYjBCIAww1wdx1xt7etv+WymkuUmSWOjLwrWFgl10aC7a9//wklr6A2xCtdGqlGz/MGkB7 2DkUpJ/IGPG3zkarNfwOFFO075BR5VmKP1/DswlFRX4NTOKRvmsauNTBUQlouJH9pH9Sma3RR ybOoJN6hVBk1SOxHfqN96Eq1rnh9/5i/wCxKoajlcgesBJSUJrkEiztPQ6XqDBEWUF1GO+F9X 1yukq2Gtgtq2iLwCDSGJIqHXrlpK/Gn6Tgn7BPYQRgHOPbsErh9K7DbUvAXkPcgo+bKjEk1+Q mVfHqJ+NeTHKzqUAZdcjGUjASviAyBTJpfPpWdRbzDk1IgJmhXTxgr0ooSZkwy2vMMqx0SfzM dDC588ndnqn60JEj2cWzZXjCpg5XGXVAD6eW0Pnvv+3R6FwJhG3+i+LsAEzHtfFSnPwt8urJh 834fldbUoYP8qr7ozCLw4agSDzWxY0L9mJpTVAC9zLy8kjRAWl8/3BeTM0bvIBhQLA0MXd8YE 5iIg9UobWp3DOYupKN2w== X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.34 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.102.4 at phobos.denx.de X-Virus-Status: Clean Carve out a function to translate a Unicode code point to an 8bit codepage. Provide a unit test for the new function. Signed-off-by: Heinrich Schuchardt --- include/charset.h | 11 ++++++++++ lib/charset.c | 28 +++++++++++++++++++++++++ lib/efi_loader/efi_unicode_collation.c | 19 +++-------------- test/unicode_ut.c | 29 ++++++++++++++++++++++++++ 4 files changed, 71 insertions(+), 16 deletions(-) -- 2.30.0 diff --git a/include/charset.h b/include/charset.h index 64ba91f791..52e7d1474e 100644 --- a/include/charset.h +++ b/include/charset.h @@ -275,4 +275,15 @@ u16 *u16_strdup(const void *src); */ uint8_t *utf16_to_utf8(uint8_t *dest, const uint16_t *src, size_t size); +/** + * utf_to_cp() - translate Unicode code point to 8bit codepage + * + * Codepoints that do not exist in the codepage are rendered as question mark. + * + * @c: pointer to Unicode code point to be translated + * @codepage: Unicode to codepage translation table + * Return: 0 on success, -ENOENT if codepoint cannot be translated + */ +int utf_to_cp(s32 *c, const u16 *codepage); + #endif /* __CHARSET_H_ */ diff --git a/lib/charset.c b/lib/charset.c index 814847d165..1345c8f9f0 100644 --- a/lib/charset.c +++ b/lib/charset.c @@ -10,6 +10,7 @@ #include #include #include +#include #include /** @@ -472,3 +473,30 @@ uint8_t *utf16_to_utf8(uint8_t *dest, const uint16_t *src, size_t size) return dest; } + +/** + * utf_to_cp() - translate Unicode code point to 8bit codepage + * + * Codepoints that do not exist in the codepage are rendered as question mark. + * + * @c: pointer to Unicode code point to be translated + * @codepage: Unicode to codepage translation table + * Return: 0 on success, -ENOENT if codepoint cannot be translated + */ +int utf_to_cp(s32 *c, const u16 *codepage) +{ + if (*c >= 0x80) { + int j; + + /* Look up codepage translation */ + for (j = 0; j < 0x80; ++j) { + if (*c == codepage[j]) { + *c = j + 0x80; + return 0; + } + } + *c = '?'; + return -ENOENT; + } + return 0; +} diff --git a/lib/efi_loader/efi_unicode_collation.c b/lib/efi_loader/efi_unicode_collation.c index bf5314c4ff..36be798f64 100644 --- a/lib/efi_loader/efi_unicode_collation.c +++ b/lib/efi_loader/efi_unicode_collation.c @@ -300,23 +300,10 @@ static bool EFIAPI efi_str_to_fat(struct efi_unicode_collation_protocol *this, break; } c = utf_to_upper(c); - if (c >= 0x80) { - int j; - - /* Look for codepage translation */ - for (j = 0; j < 0x80; ++j) { - if (c == codepage[j]) { - c = j + 0x80; - break; - } - } - if (j >= 0x80) { - c = '_'; - ret = true; - } - } else if (c && (c < 0x20 || strchr(illegal, c))) { - c = '_'; + if (utf_to_cp(&c, codepage) || + (c && (c < 0x20 || strchr(illegal, c)))) { ret = true; + c = '_'; } fat[i] = c; diff --git a/test/unicode_ut.c b/test/unicode_ut.c index 6130ef0b54..2cc6b5feff 100644 --- a/test/unicode_ut.c +++ b/test/unicode_ut.c @@ -595,6 +595,35 @@ static int unicode_test_u16_strsize(struct unit_test_state *uts) } UNICODE_TEST(unicode_test_u16_strsize); +static int unicode_test_utf_to_cp(struct unit_test_state *uts) +{ + int ret; + s32 c; + + c = '\n'; + ret = utf_to_cp(&c, codepage_437); + ut_asserteq(0, ret); + ut_asserteq('\n', c); + + c = 'a'; + ret = utf_to_cp(&c, codepage_437); + ut_asserteq(0, ret); + ut_asserteq('a', c); + + c = 0x03c4; /* Greek small letter tau */ + ret = utf_to_cp(&c, codepage_437); + ut_asserteq(0, ret); + ut_asserteq(0xe7, c); + + c = 0x03a4; /* Greek capital letter tau */ + ret = utf_to_cp(&c, codepage_437); + ut_asserteq(-ENOENT, ret); + ut_asserteq('?', c); + + return 0; +} +UNICODE_TEST(unicode_test_utf_to_cp); + #ifdef CONFIG_EFI_LOADER static int unicode_test_efi_create_indexed_name(struct unit_test_state *uts) { From patchwork Sat Feb 27 13:08:37 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Heinrich Schuchardt X-Patchwork-Id: 1445172 X-Patchwork-Delegate: xypron.glpk@gmx.de Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=2a01:238:438b:c500:173d:9f52:ddab:ee01; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; secure) header.d=gmx.net header.i=@gmx.net header.a=rsa-sha256 header.s=badeba3b8450 header.b=lXsPGKXi; dkim-atps=neutral Received: from phobos.denx.de (phobos.denx.de [IPv6:2a01:238:438b:c500:173d:9f52:ddab:ee01]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4Dnn1H3XPRz9sBJ for ; Sun, 28 Feb 2021 00:09:39 +1100 (AEDT) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id 1019E8207B; Sat, 27 Feb 2021 14:09:35 +0100 (CET) Authentication-Results: phobos.denx.de; dmarc=fail (p=none dis=none) header.from=gmx.de Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; secure) header.d=gmx.net header.i=@gmx.net header.b="lXsPGKXi"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id 4716E82047; Sat, 27 Feb 2021 14:09:15 +0100 (CET) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,FREEMAIL_FROM,RCVD_IN_MSPIKE_H2,SPF_HELO_NONE autolearn=ham autolearn_force=no version=3.4.2 Received: from mout.gmx.net (mout.gmx.net [212.227.17.20]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id EF87681FBA for ; Sat, 27 Feb 2021 14:09:05 +0100 (CET) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=gmx.de Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=xypron.glpk@gmx.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gmx.net; s=badeba3b8450; t=1614431345; bh=Vn4z/LFX95Sy+15hI/hHUBVo9DQ2vg1POxiAbWnEaBs=; h=X-UI-Sender-Class:From:To:Cc:Subject:Date:In-Reply-To:References; b=lXsPGKXi9c9sD1ai596kc4CCiCBLn+h4hrc2rWhBQSXf8WR7kIroSpa8k50g75DZo t5GlVVBPrRugb8kqMnPcpDHgZ71UBkcf/UCp/ogaLjSWaJlDFOWM6rpQyZ8QYkybXx W4kHistnIJ4nDAPcAfI0URtVfqk5OmLAB4IfQmJE= X-UI-Sender-Class: 01bb95c1-4bf8-414a-932a-4f6e2808ef9c Received: from LT02.fritz.box ([62.143.246.89]) by mail.gmx.net (mrgmx105 [212.227.17.174]) with ESMTPSA (Nemesis) id 1N49hB-1lyMgT3c9p-0105Xs; Sat, 27 Feb 2021 14:09:04 +0100 From: Heinrich Schuchardt To: Alexander Graf , Anatolij Gustschin Cc: u-boot@lists.denx.de, Heinrich Schuchardt Subject: [PATCH 3/6] lib/charset: utf8_get() should return error Date: Sat, 27 Feb 2021 14:08:37 +0100 Message-Id: <20210227130840.166193-4-xypron.glpk@gmx.de> X-Mailer: git-send-email 2.30.0 In-Reply-To: <20210227130840.166193-1-xypron.glpk@gmx.de> References: <20210227130840.166193-1-xypron.glpk@gmx.de> MIME-Version: 1.0 X-Provags-ID: V03:K1:alRset+TKqoSuYIaBBa2zSqh2EHv4e9S79z3s4FVHzBvVv6HlEz 0YLL56tqsEOi82TdBqPS0GygC5JKDW6GcSkZqLvU2FgKMsVfGVxCmz9gYw7mdD2DF9wuqqH moNPQC9cwLMZhgu5OZ4aoApZocOuwtZ+5nDkyMvKD6oXSiYoSh1vpVm2DNINgQkny15LDfx Q2s6MX1IpcR//95+/Y07A== X-UI-Out-Filterresults: notjunk:1;V03:K0:AWQe4+jJg4w=:o4DcpTsbE113WPq8gT7Sfx 92hr24Vnuk+hBcGGQhdrPi7FCZpsjpPloWEt+T+ybNTpMtxHVzdJ7HZXr2B9ZMV+wT9t+4Y4d L0tLb/mzYVNav18pUUHJ3y7craXj6ybs915yJV5aoEpccnUyHXM66Kz6qU+zPpb9nmq7Ec/Em 6H34aiAH0BcWB8RhWCNHPlun4cIW9fZfYmmkVwAKwE+9oDaGPI3EeMr+/9bjAUNLO1t4H4Qw/ 31Fj8KRYgMMMPgimuR23BFQZmlEqY2aq3l9BnZVlb7oaY7h1PUPUcZvcCtvh1ZZkPFPBLyoyZ oWv8/k9Ta6FdxxiVrEViPGlJrScuA+ZJJ3Jv+Xqo02QabNHFeRYUqg2yzgWbYmo7l5VTWgt3Y cOKN+LN+SlxAAJTmPNjXgSuzCtQv7SaXjV9j3wG1iAo/PIhwBrfI344C172D0LJ/9PPus/BXW a2SOgZLDxCoi06NOqHC2PWzZx+TIxjehnno0NVEeXqLMdkxhPFNdijfu3k2vtxdMGoNE6aJ2W QC+22cPJ/TmgrxloB9hSrtJDD9rb8xrJVTRbR8pj4G9yG0PcnJnwaMoonLN5fS8GVOcVQBIt3 53VNWf4ZxnSP3J5hghbJln9ttVbe6kpw57SmFWrErMO5wslkhuLWM7idJ5WuBl1plxtRXbbnT A6+Nqtr/IrnInHvGFqJKeKQowPlWKad6zijCqeBVaWhLD7mTFBK+io1/oo9dIrCBW9QlzFh1n 2WPK2dIu20oCrtGIW1u7vaSQFzJM5DJjZNto+AUcsE2KtLPSwb4ra+LEhB/OIoINne+78Bc7Z e3b1EU9VU+kaioCDrg5TsoaJ6cAbj6XR5dYdgLjvypwkGwzPyHVkPno4RXBRAVUF0D4XP206U szriwAB3ezrNbIed995g== X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.34 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.102.4 at phobos.denx.de X-Virus-Status: Clean utf8_get() should return an error if hitting an illegal UTF-8 sequence and not silently convert the input to a question mark. Correct utf_8() and the its unit test. console_read_unicode() now will ignore illegal UTF-8 sequences. Signed-off-by: Heinrich Schuchardt --- lib/charset.c | 25 ++++++++++++++++--------- test/unicode_ut.c | 7 +++++++ 2 files changed, 23 insertions(+), 9 deletions(-) -- 2.30.0 diff --git a/lib/charset.c b/lib/charset.c index 1345c8f9f0..946d5ee23e 100644 --- a/lib/charset.c +++ b/lib/charset.c @@ -32,7 +32,7 @@ static struct capitalization_table capitalization_table[] = * * @read_u8: - stream reader * @src: - string buffer passed to stream reader, optional - * Return: - Unicode code point + * Return: - Unicode code point, or -1 */ static int get_code(u8 (*read_u8)(void *data), void *data) { @@ -78,7 +78,7 @@ static int get_code(u8 (*read_u8)(void *data), void *data) } return ch; error: - return '?'; + return -1; } /** @@ -120,14 +120,21 @@ static u8 read_console(void *data) int console_read_unicode(s32 *code) { - if (!tstc()) { - /* No input available */ - return 1; - } + for (;;) { + s32 c; - /* Read Unicode code */ - *code = get_code(read_console, NULL); - return 0; + if (!tstc()) { + /* No input available */ + return 1; + } + + /* Read Unicode code */ + c = get_code(read_console, NULL); + if (c > 0) { + *code = c; + return 0; + } + } } s32 utf8_get(const char **src) diff --git a/test/unicode_ut.c b/test/unicode_ut.c index 2cc6b5feff..154361aea7 100644 --- a/test/unicode_ut.c +++ b/test/unicode_ut.c @@ -52,6 +52,7 @@ static const char d4[] = {0xf0, 0x90, 0x92, 0x8d, 0xf0, 0x90, 0x92, 0x96, static const char j1[] = {0x6a, 0x31, 0xa1, 0x6c, 0x00}; static const char j2[] = {0x6a, 0x32, 0xc3, 0xc3, 0x6c, 0x00}; static const char j3[] = {0x6a, 0x33, 0xf0, 0x90, 0xf0, 0x00}; +static const char j4[] = {0xa1, 0x00}; static int unicode_test_u16_strlen(struct unit_test_state *uts) { @@ -165,6 +166,12 @@ static int unicode_test_utf8_get(struct unit_test_state *uts) ut_asserteq(0x0001048d, code); ut_asserteq_ptr(s, d4 + 4); + /* Check illegal character */ + s = j4; + code = utf8_get((const char **)&s); + ut_asserteq(-1, code); + ut_asserteq_ptr(j4 + 1, s); + return 0; } UNICODE_TEST(unicode_test_utf8_get); From patchwork Sat Feb 27 13:08:38 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Heinrich Schuchardt X-Patchwork-Id: 1445174 X-Patchwork-Delegate: xypron.glpk@gmx.de Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=2a01:238:438b:c500:173d:9f52:ddab:ee01; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; secure) header.d=gmx.net header.i=@gmx.net header.a=rsa-sha256 header.s=badeba3b8450 header.b=AjM8YHaX; dkim-atps=neutral Received: from phobos.denx.de (phobos.denx.de [IPv6:2a01:238:438b:c500:173d:9f52:ddab:ee01]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4Dnn1w43fPz9sBJ for ; Sun, 28 Feb 2021 00:10:12 +1100 (AEDT) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id B4DFB8201A; Sat, 27 Feb 2021 14:10:08 +0100 (CET) Authentication-Results: phobos.denx.de; dmarc=fail (p=none dis=none) header.from=gmx.de Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; secure) header.d=gmx.net header.i=@gmx.net header.b="AjM8YHaX"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id 5E2CE81FBA; Sat, 27 Feb 2021 14:09:17 +0100 (CET) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,FREEMAIL_FROM,RCVD_IN_MSPIKE_H2,SPF_HELO_NONE autolearn=ham autolearn_force=no version=3.4.2 Received: from mout.gmx.net (mout.gmx.net [212.227.17.21]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id 2C25581FC3 for ; Sat, 27 Feb 2021 14:09:06 +0100 (CET) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=gmx.de Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=xypron.glpk@gmx.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gmx.net; s=badeba3b8450; t=1614431345; bh=e5Ksr6IKbG6PqqPmQ78QGY1mG9bfOuHNK6crIhhn1gk=; h=X-UI-Sender-Class:From:To:Cc:Subject:Date:In-Reply-To:References; b=AjM8YHaXBJelkubLwyK/Rs9NfTkrtQkuT1GE0eY1LgxuwVI7NDabkyd+lVWFOJgsV nxfJsX9I91971x2KdqhQPF0ObC3wRG2xd63xnUWkaD2udLDNSYnvuonED0/Lu+5seH QEcXJTLxCYg+c8Fw006KqZBtXwr0UglOKCeaeLtI= X-UI-Sender-Class: 01bb95c1-4bf8-414a-932a-4f6e2808ef9c Received: from LT02.fritz.box ([62.143.246.89]) by mail.gmx.net (mrgmx105 [212.227.17.174]) with ESMTPSA (Nemesis) id 1N3bSt-1lxoig0N6J-010dVi; Sat, 27 Feb 2021 14:09:05 +0100 From: Heinrich Schuchardt To: Alexander Graf , Anatolij Gustschin Cc: u-boot@lists.denx.de, Heinrich Schuchardt Subject: [PATCH 4/6] lib/charset: UTF-8 stream conversion Date: Sat, 27 Feb 2021 14:08:38 +0100 Message-Id: <20210227130840.166193-5-xypron.glpk@gmx.de> X-Mailer: git-send-email 2.30.0 In-Reply-To: <20210227130840.166193-1-xypron.glpk@gmx.de> References: <20210227130840.166193-1-xypron.glpk@gmx.de> MIME-Version: 1.0 X-Provags-ID: V03:K1:JwqacGwAvEuZI7/yuONOEri6GYLsULrN/2lISEVv00cx4ID4WVC LMWcMaastYMGrn+rzQzWi0eb27dfMrupqD0QebyVUc1qy9qmob7jpcgIZ+aQu3P26ateVvx e/B65uUeP0KlUjrc58nNcrqazNJzHV6f0fJC+LdiszAqNeBAzfPDYPl9ut5w7lv6cwkkoqh CSYPo7twG+tjzm6loSM+w== X-UI-Out-Filterresults: notjunk:1;V03:K0:d6y3c+XMUi8=:59Xls68cyg4ga8wA1/L5Vt ZSg5EAgiqcNEhArVDQqQBw00P/EZtBX/1OaFgYCcj3V32iCySxSsS+KjsPP2p2MrWpKL6pLSZ VblqVwINDDJpUj77LHyBxTVUOtNIrbsH2KvDMl7qDTD+E3eo02TJaSbUIWBOZGaWXncWj6jGF hGrMGpJG64TilmY+DM79IRP21IBDi6mpTyCppUNoumBxOVaIG4TA/OTYfO5cHvZrnZ3P2+qtk E2V9Bxrrn8SLBy84sRlRm97F7iOEpjTQ2BjZBx4kSX2juvhh3Op78PWM6YXa11qiYF/SyrpiV w99ORKlCMu2vnP4F41TzRLybwxi9pRXmZuOsPFSdHgJBeTbgymTpIbLX98r1Bz3O8hodhQDgb 7lLWX4vP5S4Uwrg+XLhrXROnW7vXpXY0DSLD9/1HwUzPTsQZvnlMc6Q4CXZ68shxb6B1V8e9W 6uoVfkkCSH+izCE926SWvzd7MLyvdanfc39grSwigGPdFUS1SyJSCKT3aL6Yip+mk8Y62XQ0O /lrWfkPn2PYu6BEQhN33MpxytqEtSHsiqckrxFNykViKDZ08PU3j69QdScCKmjp9p2Ph6qXRg 0b+tS+XwMXsvgNTurJ1fhUaC0qcuv3BRmxg2Vu4Avog5K7qKOCF85yTm6pM96rIjQpNBZkRXY BZOr+k5BA4/TWMGDed3T+OVifkPK000gASC9WTAU2oNxIFoUpaFV9IppvCOEhEXjpS21PyM3P xV6VR5SJ5GVoin23nZD7861CN1IGiPA1AmUtHFM5VRwSgVohkko7ztivyYEY5z+dGGWW5/hoc n/qkCNUd4rtjyFe8w+dtOzwqg07eXxS92wKFqbsHV19S6d4V+kQzXnyHYJd9I1TSPAAJ8EOra Y7tUp2sseuFr1JAWgoTQ== X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.34 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.102.4 at phobos.denx.de X-Virus-Status: Clean Provide functions to convert an UTF-8 stream to code page 437 or UTF-32. Add unit tests. Signed-off-by: Heinrich Schuchardt --- include/charset.h | 18 +++++++++++ lib/charset.c | 55 +++++++++++++++++++++++++++------ test/unicode_ut.c | 78 +++++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 142 insertions(+), 9 deletions(-) -- 2.30.0 diff --git a/include/charset.h b/include/charset.h index 52e7d1474e..a911160f19 100644 --- a/include/charset.h +++ b/include/charset.h @@ -286,4 +286,22 @@ uint8_t *utf16_to_utf8(uint8_t *dest, const uint16_t *src, size_t size); */ int utf_to_cp(s32 *c, const u16 *codepage); +/** + * utf8_to_cp437_stream() - convert UTF-8 stream to codepage 437 + * + * @c: next UTF-8 character to convert + * @buffer: buffer, at least 5 characters + * Return: next codepage 437 character or 0 + */ +int utf8_to_cp437_stream(u8 c, char *buffer); + +/** + * utf8_to_utf32_stream() - convert UTF-8 stream to UTF-32 + * + * @c: next UTF-8 character to convert + * @buffer: buffer, at least 5 characters + * Return: next codepage 437 character or 0 + */ +int utf8_to_utf32_stream(u8 c, char *buffer); + #endif /* __CHARSET_H_ */ diff --git a/lib/charset.c b/lib/charset.c index 946d5ee23e..f44c58d9d8 100644 --- a/lib/charset.c +++ b/lib/charset.c @@ -481,15 +481,6 @@ uint8_t *utf16_to_utf8(uint8_t *dest, const uint16_t *src, size_t size) return dest; } -/** - * utf_to_cp() - translate Unicode code point to 8bit codepage - * - * Codepoints that do not exist in the codepage are rendered as question mark. - * - * @c: pointer to Unicode code point to be translated - * @codepage: Unicode to codepage translation table - * Return: 0 on success, -ENOENT if codepoint cannot be translated - */ int utf_to_cp(s32 *c, const u16 *codepage) { if (*c >= 0x80) { @@ -507,3 +498,49 @@ int utf_to_cp(s32 *c, const u16 *codepage) } return 0; } + +int utf8_to_cp437_stream(u8 c, char *buffer) +{ + char *end; + const char *pos; + s32 s; + int ret; + + for (;;) { + pos = buffer; + end = buffer + strlen(buffer); + *end++ = c; + *end = 0; + s = utf8_get(&pos); + if (s > 0) { + *buffer = 0; + ret = utf_to_cp(&s, codepage_437); + return s; + } + if (pos == end) + return 0; + *buffer = 0; + } +} + +int utf8_to_utf32_stream(u8 c, char *buffer) +{ + char *end; + const char *pos; + s32 s; + + for (;;) { + pos = buffer; + end = buffer + strlen(buffer); + *end++ = c; + *end = 0; + s = utf8_get(&pos); + if (s > 0) { + *buffer = 0; + return s; + } + if (pos == end) + return 0; + *buffer = 0; + } +} diff --git a/test/unicode_ut.c b/test/unicode_ut.c index 154361aea7..6f6aea5f60 100644 --- a/test/unicode_ut.c +++ b/test/unicode_ut.c @@ -47,6 +47,9 @@ static const char d3[] = {0xe6, 0xbd, 0x9c, 0xe6, 0xb0, 0xb4, 0xe8, 0x89, /* Three letters translating to two utf-16 word each */ static const char d4[] = {0xf0, 0x90, 0x92, 0x8d, 0xf0, 0x90, 0x92, 0x96, 0xf0, 0x90, 0x92, 0x87, 0x00}; +/* Letter not in code page 437 */ +static const char d5[] = {0xCE, 0x92, 0x20, 0x69, 0x73, 0x20, 0x6E, 0x6F, + 0x74, 0x20, 0x42, 0x00}; /* Illegal utf-8 strings */ static const char j1[] = {0x6a, 0x31, 0xa1, 0x6c, 0x00}; @@ -631,6 +634,81 @@ static int unicode_test_utf_to_cp(struct unit_test_state *uts) } UNICODE_TEST(unicode_test_utf_to_cp); +static void utf8_to_cp437_stream_helper(const char *in, char *out) +{ + char buffer[5]; + int ret; + + *buffer = 0; + for (; *in; ++in) { + ret = utf8_to_cp437_stream(*in, buffer); + if (ret) + *out++ = ret; + } + *out = 0; +} + +static int unicode_test_utf8_to_cp437_stream(struct unit_test_state *uts) +{ + char buf[16]; + + utf8_to_cp437_stream_helper(d1, buf); + ut_asserteq_str("U-Boot", buf); + utf8_to_cp437_stream_helper(d2, buf); + ut_asserteq_str("kafb\xa0tur", buf); + utf8_to_cp437_stream_helper(d5, buf); + ut_asserteq_str("? is not B", buf); + utf8_to_cp437_stream_helper(j2, buf); + ut_asserteq_str("j2l", buf); + + return 0; +} +UNICODE_TEST(unicode_test_utf8_to_cp437_stream); + +static void utf8_to_utf32_stream_helper(const char *in, s32 *out) +{ + char buffer[5]; + int ret; + + *buffer = 0; + for (; *in; ++in) { + ret = utf8_to_utf32_stream(*in, buffer); + if (ret) + *out++ = ret; + } + *out = 0; +} + +static int unicode_test_utf8_to_utf32_stream(struct unit_test_state *uts) +{ + s32 buf[16]; + + const u32 u1[] = {0x55, 0x2D, 0x42, 0x6F, 0x6F, 0x74, 0x0000}; + const u32 u2[] = {0x6B, 0x61, 0x66, 0x62, 0xE1, 0x74, 0x75, 0x72, 0x00}; + const u32 u3[] = {0x0392, 0x20, 0x69, 0x73, 0x20, 0x6E, 0x6F, 0x74, + 0x20, 0x42, 0x00}; + const u32 u4[] = {0x6A, 0x32, 0x6C, 0x00}; + + memset(buf, 0, sizeof(buf)); + utf8_to_utf32_stream_helper(d1, buf); + ut_asserteq_mem(u1, buf, sizeof(u1)); + + memset(buf, 0, sizeof(buf)); + utf8_to_utf32_stream_helper(d2, buf); + ut_asserteq_mem(u2, buf, sizeof(u2)); + + memset(buf, 0, sizeof(buf)); + utf8_to_utf32_stream_helper(d5, buf); + ut_asserteq_mem(u3, buf, sizeof(u3)); + + memset(buf, 0, sizeof(buf)); + utf8_to_utf32_stream_helper(j2, buf); + ut_asserteq_mem(u4, buf, sizeof(u4)); + + return 0; +} +UNICODE_TEST(unicode_test_utf8_to_utf32_stream); + #ifdef CONFIG_EFI_LOADER static int unicode_test_efi_create_indexed_name(struct unit_test_state *uts) { From patchwork Sat Feb 27 13:08:39 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Heinrich Schuchardt X-Patchwork-Id: 1445176 X-Patchwork-Delegate: xypron.glpk@gmx.de Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=85.214.62.61; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; secure) header.d=gmx.net header.i=@gmx.net header.a=rsa-sha256 header.s=badeba3b8450 header.b=GkSxQ4a2; dkim-atps=neutral Received: from phobos.denx.de (phobos.denx.de [85.214.62.61]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4Dnn2L02Fhz9sBJ for ; Sun, 28 Feb 2021 00:10:33 +1100 (AEDT) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id EE74D82789; Sat, 27 Feb 2021 14:10:21 +0100 (CET) Authentication-Results: phobos.denx.de; dmarc=fail (p=none dis=none) header.from=gmx.de Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; secure) header.d=gmx.net header.i=@gmx.net header.b="GkSxQ4a2"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id 4FEFD82036; Sat, 27 Feb 2021 14:09:16 +0100 (CET) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,FREEMAIL_FROM,RCVD_IN_MSPIKE_H2,SPF_HELO_NONE autolearn=ham autolearn_force=no version=3.4.2 Received: from mout.gmx.net (mout.gmx.net [212.227.17.20]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id 4A80E81FE7 for ; Sat, 27 Feb 2021 14:09:06 +0100 (CET) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=gmx.de Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=xypron.glpk@gmx.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gmx.net; s=badeba3b8450; t=1614431345; bh=DP91S9Vg6Vr07fZkMMSPOqhn75/7g3UnEiKUlS6EauU=; h=X-UI-Sender-Class:From:To:Cc:Subject:Date:In-Reply-To:References; b=GkSxQ4a2KAaYgAntbPTnLh0sulXBEKSLPADDzEOzbRqUda/tTmr6njEY2iMf4mHot +UlwtZek9TxxbdMeAF+Nm5F6GGi5HaNuYW88D5PPh+unKxvq8E75zpVI32i17BmKZy 8H3IYgDu3ZqXgEB9Kx2e0LhAg7HnyA06mxIr6yxE= X-UI-Sender-Class: 01bb95c1-4bf8-414a-932a-4f6e2808ef9c Received: from LT02.fritz.box ([62.143.246.89]) by mail.gmx.net (mrgmx105 [212.227.17.174]) with ESMTPSA (Nemesis) id 1MS3il-1lMI2s1Mbb-00TUcE; Sat, 27 Feb 2021 14:09:05 +0100 From: Heinrich Schuchardt To: Alexander Graf , Anatolij Gustschin Cc: u-boot@lists.denx.de, Heinrich Schuchardt Subject: [PATCH 5/6] video: use int for character in putc_xy() Date: Sat, 27 Feb 2021 14:08:39 +0100 Message-Id: <20210227130840.166193-6-xypron.glpk@gmx.de> X-Mailer: git-send-email 2.30.0 In-Reply-To: <20210227130840.166193-1-xypron.glpk@gmx.de> References: <20210227130840.166193-1-xypron.glpk@gmx.de> MIME-Version: 1.0 X-Provags-ID: V03:K1:vzffYEjyHJHpMWpTAzP862l8g7CXtVsBaDvYRMwMt9HJ50zAe64 2FJGr1xtGXfeH3pVEFQv1OPe6DreRgbpDWW95A9MPOwgVI1B5qHBwY3qXhGbQ7pmE3GUmJE wkeLPIIBX1RXZra2K0pIiIxgvQyYnxdP1CU5wND8ZHyoo9Uv+Foy86On9StMUL0mSiuBK6d iDa5oX7l4kv7mmvOBtOyA== X-UI-Out-Filterresults: notjunk:1;V03:K0:JC9OQXc1brk=:N7feJeCoiiqvPV7ZIDHssc f1ZpRBUfo557vB2JggpIyjmhPbgYCJoFZZBkIQ94AgquVmG80Ytn7BUlFXqjxDHGt4/uPuUG6 AxJiNIaVrLlA7jbnmz0r3IO8GokuwdPjliyIVdH3ybRMuHh70zwqsZgOA8p/y0MqVJWI2a7FV oVvhgzLhwrEaxHGV0LSEfZKNw3cXir/8gIJi672Hj2Rwz2lPyFmKhDZBBQFuACFYu3lAqbNrD JzcuJs3+nidKg/AGRGv7VWm2AO55Q9xC+XjOBUGFquPF+u9jon/+zT0Xoaenhvyqi8W40djp+ 3yONMDKVtg0TkFBmtQjxby2F9TX0emwVcI//lfJrEh55TpWOVmN9WvodUCMylslb833pzfPV+ 2dXpo8ltCftO5Ocqwl5jwLtgZFM+h/0JZYUqQlvLP3k0bIYWX1dzDAVCyFhwnfRS8JxN+zicU +wOvFEQnGXF2I7Fs8P708IliPJDARP1MMyKQZjS23cNAnptpY2DCmrTDFWAOMz1Rmg4LQZ5C2 jjCD7sUY3h2c3CgFgBBHfzDSRf6Hyerw92QWvRoYPm7ZxwHNxSh1WN6FpPZz4Jluix/Ykxjrx W7qT3SOgabxd1UkW+ujE1CdJN1aruV6R+QwbqvTXOwrZ+8rZWvhoT8u9E5OO65dfj/xO3z9YJ +tmAwtksUDbH+/T+EcnKHFPVkFUhYDnWo+rQy6DzOo3MCu8A9DNSjOFDpowWRXaUFNQLW7EWj r2xhM2oP0+JYMKApMYvf5iANMTb7+RAwiY6N1V6Gaq8E3FXyaDEPJXGMto3WzEv7TwG3Bm8VY XQJZerB/F2xf9dfgcHAGUzhj5vds4IssKj6e1Zz+jq5EOeV17Mcybxc+FLYkaSVkTixc8WGfg AsM8LQmqSCXJqISir0Qw== X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.34 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.102.4 at phobos.denx.de X-Virus-Status: Clean The truetype console expects the character to be a Unicode code point. This value cannot be passed as char. Use int instead. Signed-off-by: Heinrich Schuchardt --- drivers/video/console_normal.c | 2 +- drivers/video/console_rotate.c | 6 +++--- drivers/video/console_truetype.c | 2 +- drivers/video/vidconsole-uclass.c | 9 +++++++++ include/video_console.h | 2 +- 5 files changed, 15 insertions(+), 6 deletions(-) -- 2.30.0 diff --git a/drivers/video/console_normal.c b/drivers/video/console_normal.c index 04f022491e..9cfd11ae41 100644 --- a/drivers/video/console_normal.c +++ b/drivers/video/console_normal.c @@ -80,7 +80,7 @@ static int console_normal_move_rows(struct udevice *dev, uint rowdst, } static int console_normal_putc_xy(struct udevice *dev, uint x_frac, uint y, - char ch) + int ch) { struct vidconsole_priv *vc_priv = dev_get_uclass_priv(dev); struct udevice *vid = dev->parent; diff --git a/drivers/video/console_rotate.c b/drivers/video/console_rotate.c index 36c8d0609d..78ca36431f 100644 --- a/drivers/video/console_rotate.c +++ b/drivers/video/console_rotate.c @@ -86,7 +86,7 @@ static int console_move_rows_1(struct udevice *dev, uint rowdst, uint rowsrc, return 0; } -static int console_putc_xy_1(struct udevice *dev, uint x_frac, uint y, char ch) +static int console_putc_xy_1(struct udevice *dev, uint x_frac, uint y, int ch) { struct vidconsole_priv *vc_priv = dev_get_uclass_priv(dev); struct udevice *vid = dev->parent; @@ -221,7 +221,7 @@ static int console_move_rows_2(struct udevice *dev, uint rowdst, uint rowsrc, return 0; } -static int console_putc_xy_2(struct udevice *dev, uint x_frac, uint y, char ch) +static int console_putc_xy_2(struct udevice *dev, uint x_frac, uint y, int ch) { struct vidconsole_priv *vc_priv = dev_get_uclass_priv(dev); struct udevice *vid = dev->parent; @@ -362,7 +362,7 @@ static int console_move_rows_3(struct udevice *dev, uint rowdst, uint rowsrc, return 0; } -static int console_putc_xy_3(struct udevice *dev, uint x_frac, uint y, char ch) +static int console_putc_xy_3(struct udevice *dev, uint x_frac, uint y, int ch) { struct vidconsole_priv *vc_priv = dev_get_uclass_priv(dev); struct udevice *vid = dev->parent; diff --git a/drivers/video/console_truetype.c b/drivers/video/console_truetype.c index fa11b3bbef..293b3a5ef4 100644 --- a/drivers/video/console_truetype.c +++ b/drivers/video/console_truetype.c @@ -198,7 +198,7 @@ static int console_truetype_move_rows(struct udevice *dev, uint rowdst, } static int console_truetype_putc_xy(struct udevice *dev, uint x, uint y, - char ch) + int ch) { struct vidconsole_priv *vc_priv = dev_get_uclass_priv(dev); struct udevice *vid = dev->parent; diff --git a/drivers/video/vidconsole-uclass.c b/drivers/video/vidconsole-uclass.c index 81b65f5aae..a355328851 100644 --- a/drivers/video/vidconsole-uclass.c +++ b/drivers/video/vidconsole-uclass.c @@ -8,6 +8,7 @@ */ #include +#include #include #include #include @@ -509,6 +510,14 @@ int vidconsole_put_char(struct udevice *dev, char ch) struct vidconsole_priv *priv = dev_get_uclass_priv(dev); int ret; + if (IS_ENABLED(CONFIG_EFI_LOADER)) { + static char buffer[5]; + + ch = utf8_to_cp437_stream(ch, buffer); + if (!ch) + return 0; + } + if (priv->escape) { vidconsole_escape_char(dev, ch); return 0; diff --git a/include/video_console.h b/include/video_console.h index 06b798ef10..5e6eb3cc81 100644 --- a/include/video_console.h +++ b/include/video_console.h @@ -106,7 +106,7 @@ struct vidconsole_ops { * if all is OK, -EAGAIN if we ran out of space on this line, other -ve * on error */ - int (*putc_xy)(struct udevice *dev, uint x_frac, uint y, char ch); + int (*putc_xy)(struct udevice *dev, uint x_frac, uint y, int ch); /** * move_rows() - Move text rows from one place to another From patchwork Sat Feb 27 13:08:40 2021 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Heinrich Schuchardt X-Patchwork-Id: 1445175 X-Patchwork-Delegate: xypron.glpk@gmx.de Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=pass (sender SPF authorized) smtp.mailfrom=lists.denx.de (client-ip=2a01:238:438b:c500:173d:9f52:ddab:ee01; helo=phobos.denx.de; envelope-from=u-boot-bounces@lists.denx.de; receiver=) Authentication-Results: ozlabs.org; dkim=pass (1024-bit key; secure) header.d=gmx.net header.i=@gmx.net header.a=rsa-sha256 header.s=badeba3b8450 header.b=l8qnd3IT; dkim-atps=neutral Received: from phobos.denx.de (phobos.denx.de [IPv6:2a01:238:438b:c500:173d:9f52:ddab:ee01]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits) key-exchange X25519 server-signature RSA-PSS (4096 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 4Dnn272CXTz9sRN for ; Sun, 28 Feb 2021 00:10:23 +1100 (AEDT) Received: from h2850616.stratoserver.net (localhost [IPv6:::1]) by phobos.denx.de (Postfix) with ESMTP id 1EACA8270D; Sat, 27 Feb 2021 14:10:16 +0100 (CET) Authentication-Results: phobos.denx.de; dmarc=fail (p=none dis=none) header.from=gmx.de Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=u-boot-bounces@lists.denx.de Authentication-Results: phobos.denx.de; dkim=pass (1024-bit key; secure) header.d=gmx.net header.i=@gmx.net header.b="l8qnd3IT"; dkim-atps=neutral Received: by phobos.denx.de (Postfix, from userid 109) id B8F2E82016; Sat, 27 Feb 2021 14:09:15 +0100 (CET) X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on phobos.denx.de X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,DKIM_SIGNED, DKIM_VALID,FREEMAIL_FROM,RCVD_IN_MSPIKE_H2,SPF_HELO_NONE autolearn=ham autolearn_force=no version=3.4.2 Received: from mout.gmx.net (mout.gmx.net [212.227.17.20]) (using TLSv1.3 with cipher TLS_AES_256_GCM_SHA384 (256/256 bits)) (No client certificate requested) by phobos.denx.de (Postfix) with ESMTPS id 7B1828204A for ; Sat, 27 Feb 2021 14:09:06 +0100 (CET) Authentication-Results: phobos.denx.de; dmarc=pass (p=none dis=none) header.from=gmx.de Authentication-Results: phobos.denx.de; spf=pass smtp.mailfrom=xypron.glpk@gmx.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gmx.net; s=badeba3b8450; t=1614431345; bh=j41U5mdUFgpf9vopfSiIgbopDKfBe0D3OAR7OZLKbzA=; h=X-UI-Sender-Class:From:To:Cc:Subject:Date:In-Reply-To:References; b=l8qnd3ITzgDY+kOz3wjINtuTxvbcxUXjK68V0q9MZAYdrdx+FVYuhUlyv5uf1NnyB brh+Gx9jWaptBZN8xWWNKVCEwPKPUukmZWNyRonwCrHWiir3zwhJJJpkcpAJmL+Nkl avuHxfPex/ohfyk8XWUHDaZXa8zQGo+awJ0EqPPM= X-UI-Sender-Class: 01bb95c1-4bf8-414a-932a-4f6e2808ef9c Received: from LT02.fritz.box ([62.143.246.89]) by mail.gmx.net (mrgmx105 [212.227.17.174]) with ESMTPSA (Nemesis) id 1M9Fjb-1lCQha2bdX-006OEQ; Sat, 27 Feb 2021 14:09:05 +0100 From: Heinrich Schuchardt To: Alexander Graf , Anatolij Gustschin Cc: u-boot@lists.denx.de, Heinrich Schuchardt Subject: [PATCH 6/6] video: support Unicode in video console Date: Sat, 27 Feb 2021 14:08:40 +0100 Message-Id: <20210227130840.166193-7-xypron.glpk@gmx.de> X-Mailer: git-send-email 2.30.0 In-Reply-To: <20210227130840.166193-1-xypron.glpk@gmx.de> References: <20210227130840.166193-1-xypron.glpk@gmx.de> MIME-Version: 1.0 X-Provags-ID: V03:K1:g74+cFYfQBIYhP5LZ4vkihVTaPRRbRKKC1rUFgGHOkH9G6S05WG pJVGgrX5vUFnZnhOn7Icl/XPNC8BOcn/+EtyO5Tz7ULCxz5ne1Ubo/+OEvivyIpior5Iz4J 52tNCwX3yLh/nmEn0pPyc3cL3Mh6/Tm+OatNU+Qq/+qGl0ZrY1m0gttHXhgqaln+WSM0wd0 LeTNJuwmA8JSx1w8O+4zw== X-UI-Out-Filterresults: notjunk:1;V03:K0:fJq/z9yLtIQ=:TD4NclRlD7SZKLwUJGE3Qt reaE7X8/dQsWTCJ14yryDs2PhJ03RmZ6W1NH1eBagRfrsFUDKs/KKnu9eZHUO/iX/UaWtVRPd 6VNIAedLIVELO1r9ZcJRFNejeBWFcgh8gsPkrmxxbk+Cw1xjgWYcotJLRKpLemohPj4OUA1yt daKH5AMhdHsbI/3mV5o9qG9X8/r4jjDQF//itZsdOK3PYhxjxZNb9CkDslAmFkWXcG2P+1XBQ Ir6AxwxP6ckiYa2Ei/L/LETfCEz0n0RBRjx5fdGYoP/qbCtuz/rQz6x0LycRuFjN4dpEKx7Zj MgsizlC400CzuS/H9zyJSJapEGt4FJH82GxI9v7ohHSs3a0sVcta4rsYxnjqmdznXedFBXkWA MNSw44rKtwePplSs7dI9PINheGcD7qXCy5a9sw/YQ2cpQJnvqhUHyBAttH0MG0q5Sa3uaWfwY VEZOGhO//BfDZAYpbUop6q++f081FeS5gOE8TqLAGnutY/AYqsBzsPkzJ6sBMPY/UYqxm8uu7 OnnjDZD5owULXM5T7vjyXr8v4sVxgRGh1uxDnu/M0U1r6iX+YA/nyEj1c7ZiEYJ1uyXAdqIPP 1cVLfgyvS2q7+j0jCL/EYQ2ojABcCZCyj69UZSCniZE1Jc1dPf1T6b/MJ+mAYaheVcTr8AbeG Xahdsu9MLcGXQ1tbuR1nM/prnBl8d8CMlavcZLa0sbnKSEOfBny8b+Tell4BUuNgS9DBcY6uY dde29WSjpeVOWynEV9YjuzOViJ2LqUV6tPrdSVjD2hSrJdrt3XPlkUwVXRjHzpk7SNV9WvkPA ByW1lXge/UqzJvpywvA3GaXdzgYnFeRuZvEvWWxqtkjUWQSepgQNdrx07/jGGzVh0TMyRsHHz KDkWsbH20F6QkhpUkf1w== X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.34 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Errors-To: u-boot-bounces@lists.denx.de Sender: "U-Boot" X-Virus-Scanned: clamav-milter 0.102.4 at phobos.denx.de X-Virus-Status: Clean UEFI programs use Unicode for console output, e.g. GRUB uses characters from the 0x2500 code block to draw a box around the menu. The TrueType console supports UTF-32. The normal console supports code page 437. We have to convert the UTF-8 stream passed to putc() accordingly. Signed-off-by: Heinrich Schuchardt --- drivers/video/vidconsole-uclass.c | 22 +++++++++++++++++----- include/video_console.h | 2 +- 2 files changed, 18 insertions(+), 6 deletions(-) -- 2.30.0 diff --git a/drivers/video/vidconsole-uclass.c b/drivers/video/vidconsole-uclass.c index a355328851..d5095f69e0 100644 --- a/drivers/video/vidconsole-uclass.c +++ b/drivers/video/vidconsole-uclass.c @@ -32,7 +32,7 @@ struct vid_rgb { #define CONFIG_CONSOLE_SCROLL_LINES 1 #endif -int vidconsole_putc_xy(struct udevice *dev, uint x, uint y, char ch) +int vidconsole_putc_xy(struct udevice *dev, uint x, uint y, int ch) { struct vidconsole_ops *ops = vidconsole_get_ops(dev); @@ -479,8 +479,14 @@ error: priv->escape = 0; } -/* Put that actual character on the screen (using the CP437 code page). */ -static int vidconsole_output_glyph(struct udevice *dev, char ch) +/* + * vidconsole_output_glyph() - put the actual character on the screen + * + * @dev: video device + * @ch: character: Unicode for Truetype, codepage 437 otherwise + * Return: 0 for success + */ +static int vidconsole_output_glyph(struct udevice *dev, int ch) { struct vidconsole_priv *priv = dev_get_uclass_priv(dev); int ret; @@ -505,17 +511,23 @@ static int vidconsole_output_glyph(struct udevice *dev, char ch) return 0; } -int vidconsole_put_char(struct udevice *dev, char ch) +int vidconsole_put_char(struct udevice *dev, char c) { struct vidconsole_priv *priv = dev_get_uclass_priv(dev); + int ch; int ret; if (IS_ENABLED(CONFIG_EFI_LOADER)) { static char buffer[5]; - ch = utf8_to_cp437_stream(ch, buffer); + if (IS_ENABLED(CONFIG_CONSOLE_TRUETYPE)) + ch = utf8_to_utf32_stream(c, buffer); + else + ch = utf8_to_cp437_stream(c, buffer); if (!ch) return 0; + } else { + ch = c; } if (priv->escape) { diff --git a/include/video_console.h b/include/video_console.h index 5e6eb3cc81..8747299d61 100644 --- a/include/video_console.h +++ b/include/video_console.h @@ -174,7 +174,7 @@ struct vidconsole_ops { * if all is OK, -EAGAIN if we ran out of space on this line, other -ve * on error */ -int vidconsole_putc_xy(struct udevice *dev, uint x, uint y, char ch); +int vidconsole_putc_xy(struct udevice *dev, uint x, uint y, int ch); /** * vidconsole_move_rows() - Move text rows from one place to another