mtd: nand: Kconfig: drop utf8 characters

The Linux Kernel Configuration system (lkc) expects 8 bit characters
only (declared in scripts/kconfig/zconf.l: %option 8bit). Fix two flaws
that cause minor glitches like broken characters in xconfig.

Reported-by: Martin Walch <walch.martin@web.de>
Signed-off-by: Wolfram Sang <wolfram@the-dreams.de>
---

Fixes bugzilla entry #43067

 drivers/mtd/nand/Kconfig |    4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

On 11/26/2012 05:07:25 PM, Wolfram Sang wrote:
> The Linux Kernel Configuration system (lkc) expects 8 bit characters
> only (declared in scripts/kconfig/zconf.l: %option 8bit).

That option contrasts with being limited to 7-bit characters, not with  
accepting UTF-8.  It may be that kconfig has problems with UTF-8, but I  
don't think this is why.

-Scott

On Tue, 2012-11-27 at 00:07 +0100, Wolfram Sang wrote:
>  config MTD_NAND_CAFE
> -	tristate "NAND support for OLPC CAFÉ chip"
> +	tristate "NAND support for OLPC CAFE chip"

Wolfram, despite of all of my respect, I have to Nack this patch. We are
livin in 21st century and the world switched to UTF-8. If a lkc has
issues with UTF8 - it is lkc issues - it should rather be fixed.

Thanks!

On Mon, 2012-11-26 at 17:18 -0600, Scott Wood wrote:
> On 11/26/2012 05:07:25 PM, Wolfram Sang wrote:
> > The Linux Kernel Configuration system (lkc) expects 8 bit characters
> > only (declared in scripts/kconfig/zconf.l: %option 8bit).
> 
> That option contrasts with being limited to 7-bit characters, not with  
> accepting UTF-8.  It may be that kconfig has problems with UTF-8, but I  
> don't think this is why.

Whatever has problems with UTF-8 - it is better to fix that instead of
hiding the problem by removing UTF-8 characters.

> Whatever has problems with UTF-8 - it is better to fix that instead of
> hiding the problem by removing UTF-8 characters.

Fair enough :)

Am Montag, 3. Dezember 2012, 15:29:13 schrieb Artem Bityutskiy:
> On Mon, 2012-11-26 at 17:18 -0600, Scott Wood wrote:
> > On 11/26/2012 05:07:25 PM, Wolfram Sang wrote:
> > > The Linux Kernel Configuration system (lkc) expects 8 bit characters
> > > only (declared in scripts/kconfig/zconf.l: %option 8bit).
> > 
> > That option contrasts with being limited to 7-bit characters, not with
> > accepting UTF-8.  It may be that kconfig has problems with UTF-8, but I
> > don't think this is why.
> 
> Whatever has problems with UTF-8 - it is better to fix that instead of
> hiding the problem by removing UTF-8 characters.

The kernel configuration system does not support multibyte characters. I have 
not found any hint that support for multibyte characters has been specified or 
taken into account. In many places throughout the configuration system, only 
single byte characters are assumed. In bug #43067

> https://bugzilla.kernel.org/show_bug.cgi?id=43067

I have attached screenshots showing a problem with utf-8 characters in the 
interactive nconfig menu.

More usage of utf-8 characters could even lead to worse problems: the flex 
scanner only allows the characters [A-Za-z0-9_] in symbol names. Other input 
will make the scanner ignore a character or refuse the input at all ("syntax 
error").

The handling of multibyte characters in string values depends on the 
configuration menu in use. menuconfig will not allow any multibyte input. When 
editing a predefined string with multibyte characters in it, things will break. 
nconfig is even worse. xconfig substitutes characters with '?'.

Character counts do not work correctly. When using many multibyte characters 
funny things happen like text lines being cut off.

To make a long story short: multibyte characters in Kconfig files lead to 
undefined behaviour. This is no implementation bug. The configuration system 
just has not been designed for processing them. So utf-8 support can not be 
achieved with an easy fix, but will need comprehensive changes.

I do not know if anyone is willing to actually make all the necessary work to 
properly support utf-8 in the configuration system. However, I suppose this 
will not happen any time soon. Therefore I suggest removing the multibyte 
characters for now. 

Regards
Martin Walch
--

On Mon, 2012-12-17 at 04:27 +0100, Martin Walch wrote:
> 
> To make a long story short: multibyte characters in Kconfig files lead to 
> undefined behaviour. This is no implementation bug. The configuration system 
> just has not been designed for processing them. So utf-8 support can not be 
> achieved with an easy fix, but will need comprehensive changes.
> 
> I do not know if anyone is willing to actually make all the necessary work to 
> properly support utf-8 in the configuration system. However, I suppose this 
> will not happen any time soon. Therefore I suggest removing the multibyte 
> characters for now. 

Sure, thanks for a detailed analysis. What I agree with is removing
non-ASCII characters in case they cause a functional or build problems,
just because, as you said, fixing Kbuild is a big task.

However, in this particular case, this is just about user seeing rubbins
in the configuration menu, which is not a big deal, I think, so I would
prefer to rather expose the issue than fix it.

On Tue, 2013-01-15 at 12:40 +0200, Artem Bityutskiy wrote:
> However, in this particular case, this is just about user seeing rubbins
> in the configuration menu, which is not a big deal, I think, so I would
> prefer to rather expose the issue than fix it.

Phrased badly. I wanted to say that in this particular case, when it is
only about not seeing UTF-8 characters in the menu, I prefer to expose
the issue rather than _hiding_ it.