| Message ID | 20170607121901.16873-1-daniel.schwierzeck@gmail.com |
|---|---|
| State | Superseded |
| Delegated to: | Simon Glass |
| Headers | show |
Hi Daniel, On 7 June 2017 at 06:19, Daniel Schwierzeck <daniel.schwierzeck@gmail.com> wrote: > binutils are using localized output. On systems with locales and special > characters this leads to exceptions like this: > > Exception in thread Thread-7: > Traceback (most recent call last): > File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner > self.run() > File "/u-boot/tools/buildman/builderthread.py", line 475, in run > self.RunJob(job) > File "/u-boot/tools/buildman/builderthread.py", line 456, in RunJob > self._WriteResult(result, job.keep_outputs) > File "/u-boot/tools/buildman/builderthread.py", line 333, in _WriteResult > print >>fd, dump_result.stdout, > UnicodeEncodeError: 'ascii' codec can't encode characters in position 75-76: ordinal not in range(128) > > Always encode the value of CommandResult::stdout to UTF-8 before > writing it to stdout or a log file. > > Signed-off-by: Daniel Schwierzeck <daniel.schwierzeck@gmail.com> > > --- > > Changes in v2: > - encode all occurences of CommandResult::stdout after spawning binutils to UTF-8 > - reworded commit message > > tools/buildman/builderthread.py | 10 +++++----- > 1 file changed, 5 insertions(+), 5 deletions(-) > I'm still seeing a rare failure, e.g.: https://travis-ci.org/sglass68/u-boot/jobs/240350621 Any thoughts on that? Regards, Simon
Am 07.06.2017 um 17:52 schrieb Simon Glass: > Hi Daniel, > > On 7 June 2017 at 06:19, Daniel Schwierzeck > <daniel.schwierzeck@gmail.com> wrote: >> binutils are using localized output. On systems with locales and special >> characters this leads to exceptions like this: >> >> Exception in thread Thread-7: >> Traceback (most recent call last): >> File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner >> self.run() >> File "/u-boot/tools/buildman/builderthread.py", line 475, in run >> self.RunJob(job) >> File "/u-boot/tools/buildman/builderthread.py", line 456, in RunJob >> self._WriteResult(result, job.keep_outputs) >> File "/u-boot/tools/buildman/builderthread.py", line 333, in _WriteResult >> print >>fd, dump_result.stdout, >> UnicodeEncodeError: 'ascii' codec can't encode characters in position 75-76: ordinal not in range(128) >> >> Always encode the value of CommandResult::stdout to UTF-8 before >> writing it to stdout or a log file. >> >> Signed-off-by: Daniel Schwierzeck <daniel.schwierzeck@gmail.com> >> >> --- >> >> Changes in v2: >> - encode all occurences of CommandResult::stdout after spawning binutils to UTF-8 >> - reworded commit message >> >> tools/buildman/builderthread.py | 10 +++++----- >> 1 file changed, 5 insertions(+), 5 deletions(-) >> > > I'm still seeing a rare failure, e.g.: > > https://travis-ci.org/sglass68/u-boot/jobs/240350621 > > Any thoughts on that? > I'll look into it
Hi Daniel, On 7 June 2017 at 10:03, Daniel Schwierzeck <daniel.schwierzeck@gmail.com> wrote: > > > Am 07.06.2017 um 17:52 schrieb Simon Glass: >> Hi Daniel, >> >> On 7 June 2017 at 06:19, Daniel Schwierzeck >> <daniel.schwierzeck@gmail.com> wrote: >>> binutils are using localized output. On systems with locales and special >>> characters this leads to exceptions like this: >>> >>> Exception in thread Thread-7: >>> Traceback (most recent call last): >>> File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner >>> self.run() >>> File "/u-boot/tools/buildman/builderthread.py", line 475, in run >>> self.RunJob(job) >>> File "/u-boot/tools/buildman/builderthread.py", line 456, in RunJob >>> self._WriteResult(result, job.keep_outputs) >>> File "/u-boot/tools/buildman/builderthread.py", line 333, in _WriteResult >>> print >>fd, dump_result.stdout, >>> UnicodeEncodeError: 'ascii' codec can't encode characters in position 75-76: ordinal not in range(128) >>> >>> Always encode the value of CommandResult::stdout to UTF-8 before >>> writing it to stdout or a log file. >>> >>> Signed-off-by: Daniel Schwierzeck <daniel.schwierzeck@gmail.com> >>> >>> --- >>> >>> Changes in v2: >>> - encode all occurences of CommandResult::stdout after spawning binutils to UTF-8 >>> - reworded commit message >>> >>> tools/buildman/builderthread.py | 10 +++++----- >>> 1 file changed, 5 insertions(+), 5 deletions(-) >>> >> >> I'm still seeing a rare failure, e.g.: >> >> https://travis-ci.org/sglass68/u-boot/jobs/240350621 >> >> Any thoughts on that? >> > > I'll look into it Thanks. I believe the two new patches you sent supersede this one. Regards, Simon
Am 09.06.2017 um 14:27 schrieb Simon Glass: > Hi Daniel, > > On 7 June 2017 at 10:03, Daniel Schwierzeck > <daniel.schwierzeck@gmail.com> wrote: >> >> >> Am 07.06.2017 um 17:52 schrieb Simon Glass: >>> Hi Daniel, >>> >>> On 7 June 2017 at 06:19, Daniel Schwierzeck >>> <daniel.schwierzeck@gmail.com> wrote: >>>> binutils are using localized output. On systems with locales and special >>>> characters this leads to exceptions like this: >>>> >>>> Exception in thread Thread-7: >>>> Traceback (most recent call last): >>>> File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner >>>> self.run() >>>> File "/u-boot/tools/buildman/builderthread.py", line 475, in run >>>> self.RunJob(job) >>>> File "/u-boot/tools/buildman/builderthread.py", line 456, in RunJob >>>> self._WriteResult(result, job.keep_outputs) >>>> File "/u-boot/tools/buildman/builderthread.py", line 333, in _WriteResult >>>> print >>fd, dump_result.stdout, >>>> UnicodeEncodeError: 'ascii' codec can't encode characters in position 75-76: ordinal not in range(128) >>>> >>>> Always encode the value of CommandResult::stdout to UTF-8 before >>>> writing it to stdout or a log file. >>>> >>>> Signed-off-by: Daniel Schwierzeck <daniel.schwierzeck@gmail.com> >>>> >>>> --- >>>> >>>> Changes in v2: >>>> - encode all occurences of CommandResult::stdout after spawning binutils to UTF-8 >>>> - reworded commit message >>>> >>>> tools/buildman/builderthread.py | 10 +++++----- >>>> 1 file changed, 5 insertions(+), 5 deletions(-) >>>> >>> >>> I'm still seeing a rare failure, e.g.: >>> >>> https://travis-ci.org/sglass68/u-boot/jobs/240350621 >>> >>> Any thoughts on that? >>> >> >> I'll look into it > > Thanks. I believe the two new patches you sent supersede this one. > yes, this one is superseded.
diff --git a/tools/buildman/builderthread.py b/tools/buildman/builderthread.py index acaf5007f5..4ca2f2eb90 100644 --- a/tools/buildman/builderthread.py +++ b/tools/buildman/builderthread.py @@ -280,13 +280,13 @@ class BuilderThread(threading.Thread): outfile = os.path.join(build_dir, 'log') with open(outfile, 'w') as fd: if result.stdout: - fd.write(result.stdout.encode('latin-1', 'ignore')) + fd.write(result.stdout.encode('UTF-8', 'ignore')) errfile = self.builder.GetErrFile(result.commit_upto, result.brd.target) if result.stderr: with open(errfile, 'w') as fd: - fd.write(result.stderr.encode('latin-1', 'ignore')) + fd.write(result.stderr.encode('UTF-8', 'ignore')) elif os.path.exists(errfile): os.remove(errfile) @@ -319,7 +319,7 @@ class BuilderThread(threading.Thread): nm = self.builder.GetFuncSizesFile(result.commit_upto, result.brd.target, fname) with open(nm, 'w') as fd: - print >>fd, nm_result.stdout, + print >>fd, nm_result.stdout.encode('UTF-8'), cmd = ['%sobjdump' % self.toolchain.cross, '-h', fname] dump_result = command.RunPipe([cmd], capture=True, @@ -330,7 +330,7 @@ class BuilderThread(threading.Thread): objdump = self.builder.GetObjdumpFile(result.commit_upto, result.brd.target, fname) with open(objdump, 'w') as fd: - print >>fd, dump_result.stdout, + print >>fd, dump_result.stdout.encode('UTF-8'), for line in dump_result.stdout.splitlines(): fields = line.split() if len(fields) > 5 and fields[1] == '.rodata': @@ -352,7 +352,7 @@ class BuilderThread(threading.Thread): sizes = self.builder.GetSizesFile(result.commit_upto, result.brd.target) with open(sizes, 'w') as fd: - print >>fd, '\n'.join(lines) + print >>fd, '\n'.join(lines).encode('UTF-8') # Write out the configuration files, with a special case for SPL for dirname in ['', 'spl', 'tpl']:
binutils are using localized output. On systems with locales and special characters this leads to exceptions like this: Exception in thread Thread-7: Traceback (most recent call last): File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner self.run() File "/u-boot/tools/buildman/builderthread.py", line 475, in run self.RunJob(job) File "/u-boot/tools/buildman/builderthread.py", line 456, in RunJob self._WriteResult(result, job.keep_outputs) File "/u-boot/tools/buildman/builderthread.py", line 333, in _WriteResult print >>fd, dump_result.stdout, UnicodeEncodeError: 'ascii' codec can't encode characters in position 75-76: ordinal not in range(128) Always encode the value of CommandResult::stdout to UTF-8 before writing it to stdout or a log file. Signed-off-by: Daniel Schwierzeck <daniel.schwierzeck@gmail.com> --- Changes in v2: - encode all occurences of CommandResult::stdout after spawning binutils to UTF-8 - reworded commit message tools/buildman/builderthread.py | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)