diff mbox

[U-Boot,10/13] sunxi: Make the fastboot buffer larger

Message ID 1441032373-16992-11-git-send-email-maxime.ripard@free-electrons.com
State Superseded
Delegated to: Tom Rini
Headers show

Commit Message

Maxime Ripard Aug. 31, 2015, 2:46 p.m. UTC
When using fastboot and flashing a larger image such as the main partition
of a system, the current 32MB limit for the buffer is quite small.

Increase it to something that looks decent for such a use case.

Signed-off-by: Maxime Ripard <maxime.ripard@free-electrons.com>
---
 include/configs/sunxi-common.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

Comments

Hans de Goede Aug. 31, 2015, 3:01 p.m. UTC | #1
Hi,

On 31-08-15 16:46, Maxime Ripard wrote:
> When using fastboot and flashing a larger image such as the main partition
> of a system, the current 32MB limit for the buffer is quite small.
>
> Increase it to something that looks decent for such a use case.
>
> Signed-off-by: Maxime Ripard <maxime.ripard@free-electrons.com>
> ---
>   include/configs/sunxi-common.h | 2 +-
>   1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/include/configs/sunxi-common.h b/include/configs/sunxi-common.h
> index 1abf73c31179..710521c617f5 100644
> --- a/include/configs/sunxi-common.h
> +++ b/include/configs/sunxi-common.h
> @@ -363,7 +363,7 @@ extern int soft_i2c_gpio_scl;
>   #ifdef CONFIG_USB_FUNCTION_FASTBOOT
>   #define CONFIG_CMD_FASTBOOT
>   #define CONFIG_FASTBOOT_BUF_ADDR	CONFIG_SYS_LOAD_ADDR
> -#define CONFIG_FASTBOOT_BUF_SIZE	0x2000000
> +#define CONFIG_FASTBOOT_BUF_SIZE	(256 << 20)

Hmm, where / how does this get allocated? On some boards we only
have 256M RAM, so this is not going to fit ... also if this comes
out of the heap, the current heap is only 4M and the wip sunxi
nand patches boost it to 64 (I still need to verify this works on
a 256M board, this may need a tweak to bootm_size to make sure
the bootm code does not try to put the kernel where it conflicts
with the heap ...).

Regards,

Hans
Rob Herring (Arm) Aug. 31, 2015, 7:17 p.m. UTC | #2
On Mon, Aug 31, 2015 at 10:01 AM, Hans de Goede <hdegoede@redhat.com> wrote:
> Hi,
>
> On 31-08-15 16:46, Maxime Ripard wrote:
>>
>> When using fastboot and flashing a larger image such as the main partition
>> of a system, the current 32MB limit for the buffer is quite small.
>>
>> Increase it to something that looks decent for such a use case.
>>
>> Signed-off-by: Maxime Ripard <maxime.ripard@free-electrons.com>
>> ---
>>   include/configs/sunxi-common.h | 2 +-
>>   1 file changed, 1 insertion(+), 1 deletion(-)
>>
>> diff --git a/include/configs/sunxi-common.h
>> b/include/configs/sunxi-common.h
>> index 1abf73c31179..710521c617f5 100644
>> --- a/include/configs/sunxi-common.h
>> +++ b/include/configs/sunxi-common.h
>> @@ -363,7 +363,7 @@ extern int soft_i2c_gpio_scl;
>>   #ifdef CONFIG_USB_FUNCTION_FASTBOOT
>>   #define CONFIG_CMD_FASTBOOT
>>   #define CONFIG_FASTBOOT_BUF_ADDR      CONFIG_SYS_LOAD_ADDR
>> -#define CONFIG_FASTBOOT_BUF_SIZE       0x2000000
>> +#define CONFIG_FASTBOOT_BUF_SIZE       (256 << 20)
>
>
> Hmm, where / how does this get allocated? On some boards we only
> have 256M RAM, so this is not going to fit ... also if this comes
> out of the heap, the current heap is only 4M and the wip sunxi
> nand patches boost it to 64 (I still need to verify this works on
> a 256M board, this may need a tweak to bootm_size to make sure
> the bootm code does not try to put the kernel where it conflicts
> with the heap ...).

I don't think this needs to be so big with current fastboot tool. It
will break up the files if needed. However, IIRC this only works for
sparse images, so I think this needs to be sized large enough for your
biggest bootimage.

Rob
Maxime Ripard Sept. 1, 2015, 7:05 a.m. UTC | #3
Hi,

On Mon, Aug 31, 2015 at 05:01:42PM +0200, Hans de Goede wrote:
> On 31-08-15 16:46, Maxime Ripard wrote:
> >When using fastboot and flashing a larger image such as the main partition
> >of a system, the current 32MB limit for the buffer is quite small.
> >
> >Increase it to something that looks decent for such a use case.
> >
> >Signed-off-by: Maxime Ripard <maxime.ripard@free-electrons.com>
> >---
> >  include/configs/sunxi-common.h | 2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
> >
> >diff --git a/include/configs/sunxi-common.h b/include/configs/sunxi-common.h
> >index 1abf73c31179..710521c617f5 100644
> >--- a/include/configs/sunxi-common.h
> >+++ b/include/configs/sunxi-common.h
> >@@ -363,7 +363,7 @@ extern int soft_i2c_gpio_scl;
> >  #ifdef CONFIG_USB_FUNCTION_FASTBOOT
> >  #define CONFIG_CMD_FASTBOOT
> >  #define CONFIG_FASTBOOT_BUF_ADDR	CONFIG_SYS_LOAD_ADDR
> >-#define CONFIG_FASTBOOT_BUF_SIZE	0x2000000
> >+#define CONFIG_FASTBOOT_BUF_SIZE	(256 << 20)
> 
> Hmm, where / how does this get allocated? On some boards we only
> have 256M RAM, so this is not going to fit ... also if this comes
> out of the heap, the current heap is only 4M and the wip sunxi
> nand patches boost it to 64 (I still need to verify this works on
> a 256M board, this may need a tweak to bootm_size to make sure
> the bootm code does not try to put the kernel where it conflicts
> with the heap ...).

It's not allocated, it just uses the RAM directly, starting at the
offset CONFIG_FASTBOOT_BUF_ADDR (0x42000000 in our case), just like
any *load function for example.

The only thing we have to make sure is that we won't overwrite U-boot
itself, which will be an issue on those 256MB boards...

Maxime
Maxime Ripard Sept. 1, 2015, 7:14 a.m. UTC | #4
On Mon, Aug 31, 2015 at 02:17:50PM -0500, Rob Herring wrote:
> On Mon, Aug 31, 2015 at 10:01 AM, Hans de Goede <hdegoede@redhat.com> wrote:
> > Hi,
> >
> > On 31-08-15 16:46, Maxime Ripard wrote:
> >>
> >> When using fastboot and flashing a larger image such as the main partition
> >> of a system, the current 32MB limit for the buffer is quite small.
> >>
> >> Increase it to something that looks decent for such a use case.
> >>
> >> Signed-off-by: Maxime Ripard <maxime.ripard@free-electrons.com>
> >> ---
> >>   include/configs/sunxi-common.h | 2 +-
> >>   1 file changed, 1 insertion(+), 1 deletion(-)
> >>
> >> diff --git a/include/configs/sunxi-common.h
> >> b/include/configs/sunxi-common.h
> >> index 1abf73c31179..710521c617f5 100644
> >> --- a/include/configs/sunxi-common.h
> >> +++ b/include/configs/sunxi-common.h
> >> @@ -363,7 +363,7 @@ extern int soft_i2c_gpio_scl;
> >>   #ifdef CONFIG_USB_FUNCTION_FASTBOOT
> >>   #define CONFIG_CMD_FASTBOOT
> >>   #define CONFIG_FASTBOOT_BUF_ADDR      CONFIG_SYS_LOAD_ADDR
> >> -#define CONFIG_FASTBOOT_BUF_SIZE       0x2000000
> >> +#define CONFIG_FASTBOOT_BUF_SIZE       (256 << 20)
> >
> >
> > Hmm, where / how does this get allocated? On some boards we only
> > have 256M RAM, so this is not going to fit ... also if this comes
> > out of the heap, the current heap is only 4M and the wip sunxi
> > nand patches boost it to 64 (I still need to verify this works on
> > a 256M board, this may need a tweak to bootm_size to make sure
> > the bootm code does not try to put the kernel where it conflicts
> > with the heap ...).
> 
> I don't think this needs to be so big with current fastboot tool. It
> will break up the files if needed. However, IIRC this only works for
> sparse images, so I think this needs to be sized large enough for your
> biggest bootimage.

Hmm, interesting.

Do you know how it works exactly ? Are we expected to just go on with
writing data at the offset we previously stopped in such a case? I
don't think we support that currently.

Maxime
Ian Campbell Sept. 1, 2015, 7:22 a.m. UTC | #5
On Mon, 2015-08-31 at 16:46 +0200, Maxime Ripard wrote:
> When using fastboot and flashing a larger image such as the main partition
> of a system, the current 32MB limit for the buffer is quite small.

(Apart from rooting/rescuing the odd phone I'm completely unfamiliar
with fastboot, so sorry if this is all obvious).
 
The main partition of a system these days is measured in GB, I think.
So why does going from 32MB to 256MB for the buffer make a useful
difference?

Is there some enormous per-buffer overhead which needs to be amortised?
Or is something else going on?

IOW what is the practical impact of this change?

> Increase it to something that looks decent for such a use case.
> 
> Signed-off-by: Maxime Ripard <maxime.ripard@free-electrons.com>
> ---
>  include/configs/sunxi-common.h | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/include/configs/sunxi-common.h b/include/configs/sunxi
> -common.h
> index 1abf73c31179..710521c617f5 100644
> --- a/include/configs/sunxi-common.h
> +++ b/include/configs/sunxi-common.h
> @@ -363,7 +363,7 @@ extern int soft_i2c_gpio_scl;
>  #ifdef CONFIG_USB_FUNCTION_FASTBOOT
>  #define CONFIG_CMD_FASTBOOT
>  #define CONFIG_FASTBOOT_BUF_ADDR	CONFIG_SYS_LOAD_ADDR
> -#define CONFIG_FASTBOOT_BUF_SIZE	0x2000000
> +#define CONFIG_FASTBOOT_BUF_SIZE	(256 << 20)
>  
>  #define CONFIG_FASTBOOT_FLASH
>  #define CONFIG_FASTBOOT_FLASH_MMC_DEV	0
Siarhei Siamashka Sept. 1, 2015, 7:44 a.m. UTC | #6
On Tue, 01 Sep 2015 08:22:04 +0100
Ian Campbell <ijc@hellion.org.uk> wrote:

> On Mon, 2015-08-31 at 16:46 +0200, Maxime Ripard wrote:
> > When using fastboot and flashing a larger image such as the main partition
> > of a system, the current 32MB limit for the buffer is quite small.
> 
> (Apart from rooting/rescuing the odd phone I'm completely unfamiliar
> with fastboot, so sorry if this is all obvious).
>  
> The main partition of a system these days is measured in GB, I think.
> So why does going from 32MB to 256MB for the buffer make a useful
> difference?
> 
> Is there some enormous per-buffer overhead which needs to be amortised?
> Or is something else going on?
> 
> IOW what is the practical impact of this change?

I don't know what are Maxime's plans. But if fastboot is fast and
can load the kernel and initrd to the device over USB, then it
becomes a useful alternative to using FEL for loading kernel.

FEL is implemented by BROM, but it is not particularly fast and
the transfer speed varies for different SoC variants:
    http://linux-sunxi.org/FEL/USBBoot#SoC_support_status

Somebody might want to evaluate the use of fastboot for loading kernel
with initrd and document it at http://linux-sunxi.org/FEL/USBBoot

FEL is still needed for loading U-Boot over USB on sunxi hardware, but
U-Boot size is relatively small compared to the kernel, and especially
initrd.
Maxime Ripard Sept. 1, 2015, 7:57 a.m. UTC | #7
Hi Ian,

On Tue, Sep 01, 2015 at 08:22:04AM +0100, Ian Campbell wrote:
> On Mon, 2015-08-31 at 16:46 +0200, Maxime Ripard wrote:
> > When using fastboot and flashing a larger image such as the main partition
> > of a system, the current 32MB limit for the buffer is quite small.
> 
> (Apart from rooting/rescuing the odd phone I'm completely unfamiliar
> with fastboot, so sorry if this is all obvious).
>  
> The main partition of a system these days is measured in GB, I think.
> So why does going from 32MB to 256MB for the buffer make a useful
> difference?
> 
> Is there some enormous per-buffer overhead which needs to be amortised?
> Or is something else going on?
> 
> IOW what is the practical impact of this change?

It really depends on the image content you're trying to flash, but
there's two things to consider:

  - On a NAND device, you will just flash the actual content, but not
    the extra empty space in the partition, and the FS will
    automatically grow the first time you mount it.

  - If there's some blocks that are filled with the same content, the
    sparse image format I was refactoring earlier in that patchset
    will be able to compress it to 16 bytes (iirc), instead of a full
    block (512 bytes or a page, depending on whether you use an MMC or
    a NAND). On NAND images, there's often a lot of pages filled with
    0xff, which fall into that category, I'd expect it to be similar
    on block partitions for the extra space.

The current use case we have for example is to flash a 4GB partition
with a debian system, in a UBI partition. Even though it's a decent
system, storage size-wise, the file actually transfered and stored in
the memory is around 210MB once compressed, and 270MB if not
compressed.

Of course, this will all depend on the ratio between the empty space
and the files themselves, and what files you actually have there, but
while 32MB is definitely useless, 256MB is already a decent size.

Maxime
Siarhei Siamashka Sept. 1, 2015, 8:02 a.m. UTC | #8
On Mon, 31 Aug 2015 17:01:42 +0200
Hans de Goede <hdegoede@redhat.com> wrote:

> Hi,
> 
> On 31-08-15 16:46, Maxime Ripard wrote:
> > When using fastboot and flashing a larger image such as the main partition
> > of a system, the current 32MB limit for the buffer is quite small.
> >
> > Increase it to something that looks decent for such a use case.
> >
> > Signed-off-by: Maxime Ripard <maxime.ripard@free-electrons.com>
> > ---
> >   include/configs/sunxi-common.h | 2 +-
> >   1 file changed, 1 insertion(+), 1 deletion(-)
> >
> > diff --git a/include/configs/sunxi-common.h b/include/configs/sunxi-common.h
> > index 1abf73c31179..710521c617f5 100644
> > --- a/include/configs/sunxi-common.h
> > +++ b/include/configs/sunxi-common.h
> > @@ -363,7 +363,7 @@ extern int soft_i2c_gpio_scl;
> >   #ifdef CONFIG_USB_FUNCTION_FASTBOOT
> >   #define CONFIG_CMD_FASTBOOT
> >   #define CONFIG_FASTBOOT_BUF_ADDR	CONFIG_SYS_LOAD_ADDR
> > -#define CONFIG_FASTBOOT_BUF_SIZE	0x2000000
> > +#define CONFIG_FASTBOOT_BUF_SIZE	(256 << 20)
> 
> Hmm, where / how does this get allocated? On some boards we only
> have 256M RAM, so this is not going to fit ... also if this comes
> out of the heap, the current heap is only 4M and the wip sunxi
> nand patches boost it to 64 (I still need to verify this works on
> a 256M board, this may need a tweak to bootm_size to make sure
> the bootm code does not try to put the kernel where it conflicts
> with the heap ...).

Can this be eventually improved to become a dynamic limit (depending
on the RAM size available on the device) instead of the "one size fits
all" hardcoded define approach?
Maxime Ripard Sept. 1, 2015, 8:11 a.m. UTC | #9
On Tue, Sep 01, 2015 at 10:44:58AM +0300, Siarhei Siamashka wrote:
> On Tue, 01 Sep 2015 08:22:04 +0100
> Ian Campbell <ijc@hellion.org.uk> wrote:
> 
> > On Mon, 2015-08-31 at 16:46 +0200, Maxime Ripard wrote:
> > > When using fastboot and flashing a larger image such as the main partition
> > > of a system, the current 32MB limit for the buffer is quite small.
> > 
> > (Apart from rooting/rescuing the odd phone I'm completely unfamiliar
> > with fastboot, so sorry if this is all obvious).
> >  
> > The main partition of a system these days is measured in GB, I think.
> > So why does going from 32MB to 256MB for the buffer make a useful
> > difference?
> > 
> > Is there some enormous per-buffer overhead which needs to be amortised?
> > Or is something else going on?
> > 
> > IOW what is the practical impact of this change?
> 
> I don't know what are Maxime's plans. But if fastboot is fast and
> can load the kernel and initrd to the device over USB, then it
> becomes a useful alternative to using FEL for loading kernel.

This serie is mostly to use fastboot to flash partition images to the
storage device. We're using it to program the flash the first time.

It's indeed significantly faster than using fel to do so (by a
guesstimate, something around 30-50%), and is more pratical in the
sense that you can later on reflash only a single partition, all of
that through USB.

However, you can also use it to simply boot a kernel, without touching
any of the device storage. The main drawback with that is that the DT
has to be appended, which prevents the use of simplefb or psci in our
case...

I tried to amend the android boot image format that fastboot sends
when doing so to add a section for the DT, but gave up when having to
deal with the image parser in u-boot.

Maxime
Hans de Goede Sept. 1, 2015, 8:59 a.m. UTC | #10
Hi,

On 01-09-15 09:05, Maxime Ripard wrote:
> Hi,
>
> On Mon, Aug 31, 2015 at 05:01:42PM +0200, Hans de Goede wrote:
>> On 31-08-15 16:46, Maxime Ripard wrote:
>>> When using fastboot and flashing a larger image such as the main partition
>>> of a system, the current 32MB limit for the buffer is quite small.
>>>
>>> Increase it to something that looks decent for such a use case.
>>>
>>> Signed-off-by: Maxime Ripard <maxime.ripard@free-electrons.com>
>>> ---
>>>   include/configs/sunxi-common.h | 2 +-
>>>   1 file changed, 1 insertion(+), 1 deletion(-)
>>>
>>> diff --git a/include/configs/sunxi-common.h b/include/configs/sunxi-common.h
>>> index 1abf73c31179..710521c617f5 100644
>>> --- a/include/configs/sunxi-common.h
>>> +++ b/include/configs/sunxi-common.h
>>> @@ -363,7 +363,7 @@ extern int soft_i2c_gpio_scl;
>>>   #ifdef CONFIG_USB_FUNCTION_FASTBOOT
>>>   #define CONFIG_CMD_FASTBOOT
>>>   #define CONFIG_FASTBOOT_BUF_ADDR	CONFIG_SYS_LOAD_ADDR
>>> -#define CONFIG_FASTBOOT_BUF_SIZE	0x2000000
>>> +#define CONFIG_FASTBOOT_BUF_SIZE	(256 << 20)
>>
>> Hmm, where / how does this get allocated? On some boards we only
>> have 256M RAM, so this is not going to fit ... also if this comes
>> out of the heap, the current heap is only 4M and the wip sunxi
>> nand patches boost it to 64 (I still need to verify this works on
>> a 256M board, this may need a tweak to bootm_size to make sure
>> the bootm code does not try to put the kernel where it conflicts
>> with the heap ...).
>
> It's not allocated, it just uses the RAM directly, starting at the
> offset CONFIG_FASTBOOT_BUF_ADDR (0x42000000 in our case), just like
> any *load function for example.
>
> The only thing we have to make sure is that we won't overwrite U-boot
> itself, which will be an issue on those 256MB boards...

Well the only 256M board is the A13-OLinuXino-MICRO, all other boards
have at least 512M and the A13-OLinuXino-MICRO does not have nand,
so I guess we do not really need to worry about this.

Regards,

Hans
Maxime Ripard Sept. 3, 2015, 9:43 p.m. UTC | #11
On Tue, Sep 01, 2015 at 10:59:52AM +0200, Hans de Goede wrote:
> Hi,
> 
> On 01-09-15 09:05, Maxime Ripard wrote:
> >Hi,
> >
> >On Mon, Aug 31, 2015 at 05:01:42PM +0200, Hans de Goede wrote:
> >>On 31-08-15 16:46, Maxime Ripard wrote:
> >>>When using fastboot and flashing a larger image such as the main partition
> >>>of a system, the current 32MB limit for the buffer is quite small.
> >>>
> >>>Increase it to something that looks decent for such a use case.
> >>>
> >>>Signed-off-by: Maxime Ripard <maxime.ripard@free-electrons.com>
> >>>---
> >>>  include/configs/sunxi-common.h | 2 +-
> >>>  1 file changed, 1 insertion(+), 1 deletion(-)
> >>>
> >>>diff --git a/include/configs/sunxi-common.h b/include/configs/sunxi-common.h
> >>>index 1abf73c31179..710521c617f5 100644
> >>>--- a/include/configs/sunxi-common.h
> >>>+++ b/include/configs/sunxi-common.h
> >>>@@ -363,7 +363,7 @@ extern int soft_i2c_gpio_scl;
> >>>  #ifdef CONFIG_USB_FUNCTION_FASTBOOT
> >>>  #define CONFIG_CMD_FASTBOOT
> >>>  #define CONFIG_FASTBOOT_BUF_ADDR	CONFIG_SYS_LOAD_ADDR
> >>>-#define CONFIG_FASTBOOT_BUF_SIZE	0x2000000
> >>>+#define CONFIG_FASTBOOT_BUF_SIZE	(256 << 20)
> >>
> >>Hmm, where / how does this get allocated? On some boards we only
> >>have 256M RAM, so this is not going to fit ... also if this comes
> >>out of the heap, the current heap is only 4M and the wip sunxi
> >>nand patches boost it to 64 (I still need to verify this works on
> >>a 256M board, this may need a tweak to bootm_size to make sure
> >>the bootm code does not try to put the kernel where it conflicts
> >>with the heap ...).
> >
> >It's not allocated, it just uses the RAM directly, starting at the
> >offset CONFIG_FASTBOOT_BUF_ADDR (0x42000000 in our case), just like
> >any *load function for example.
> >
> >The only thing we have to make sure is that we won't overwrite U-boot
> >itself, which will be an issue on those 256MB boards...
> 
> Well the only 256M board is the A13-OLinuXino-MICRO, all other boards
> have at least 512M and the A13-OLinuXino-MICRO does not have nand,
> so I guess we do not really need to worry about this.

You could use fastboot to flash something on the MMC, so I guess it is
something to worry about :)

Rob was kind of saying that such a huge value wasn't needed, so I
guess we can simply drop that patch.

Maxime
Tom Rini Sept. 4, 2015, 4:59 p.m. UTC | #12
On Tue, Sep 01, 2015 at 09:57:16AM +0200, Maxime Ripard wrote:

[snip]
> Of course, this will all depend on the ratio between the empty space
> and the files themselves, and what files you actually have there, but
> while 32MB is definitely useless, 256MB is already a decent size.

So there's not any automatic chunking and writing of the image?  Maybe
I'm confusing this with DFU and/or my wishlist/imagination...
Tom Rini Sept. 4, 2015, 5:02 p.m. UTC | #13
On Tue, Sep 01, 2015 at 10:44:58AM +0300, Siarhei Siamashka wrote:

> On Tue, 01 Sep 2015 08:22:04 +0100
> Ian Campbell <ijc@hellion.org.uk> wrote:
> 
> > On Mon, 2015-08-31 at 16:46 +0200, Maxime Ripard wrote:
> > > When using fastboot and flashing a larger image such as the main partition
> > > of a system, the current 32MB limit for the buffer is quite small.
> > 
> > (Apart from rooting/rescuing the odd phone I'm completely unfamiliar
> > with fastboot, so sorry if this is all obvious).
> >  
> > The main partition of a system these days is measured in GB, I think.
> > So why does going from 32MB to 256MB for the buffer make a useful
> > difference?
> > 
> > Is there some enormous per-buffer overhead which needs to be amortised?
> > Or is something else going on?
> > 
> > IOW what is the practical impact of this change?
> 
> I don't know what are Maxime's plans. But if fastboot is fast and
> can load the kernel and initrd to the device over USB, then it
> becomes a useful alternative to using FEL for loading kernel.

This is in fact a common use case for development.  Other platforms /
SoCs do use fastboot in non-Android development as how to pass in a new
testing kernel / initrd / device tree combo.
Maxime Ripard Sept. 6, 2015, 11:22 a.m. UTC | #14
On Fri, Sep 04, 2015 at 12:59:24PM -0400, Tom Rini wrote:
> On Tue, Sep 01, 2015 at 09:57:16AM +0200, Maxime Ripard wrote:
> 
> [snip]
> > Of course, this will all depend on the ratio between the empty space
> > and the files themselves, and what files you actually have there, but
> > while 32MB is definitely useless, 256MB is already a decent size.
> 
> So there's not any automatic chunking and writing of the image?  Maybe
> I'm confusing this with DFU and/or my wishlist/imagination...

Apparently, Rob was saying there was some, but I'm not sure we support
that yet, I'd need to look into it.

Maxime
Maxime Ripard Sept. 6, 2015, 11:23 a.m. UTC | #15
On Fri, Sep 04, 2015 at 01:02:49PM -0400, Tom Rini wrote:
> On Tue, Sep 01, 2015 at 10:44:58AM +0300, Siarhei Siamashka wrote:
> 
> > On Tue, 01 Sep 2015 08:22:04 +0100
> > Ian Campbell <ijc@hellion.org.uk> wrote:
> > 
> > > On Mon, 2015-08-31 at 16:46 +0200, Maxime Ripard wrote:
> > > > When using fastboot and flashing a larger image such as the main partition
> > > > of a system, the current 32MB limit for the buffer is quite small.
> > > 
> > > (Apart from rooting/rescuing the odd phone I'm completely unfamiliar
> > > with fastboot, so sorry if this is all obvious).
> > >  
> > > The main partition of a system these days is measured in GB, I think.
> > > So why does going from 32MB to 256MB for the buffer make a useful
> > > difference?
> > > 
> > > Is there some enormous per-buffer overhead which needs to be amortised?
> > > Or is something else going on?
> > > 
> > > IOW what is the practical impact of this change?
> > 
> > I don't know what are Maxime's plans. But if fastboot is fast and
> > can load the kernel and initrd to the device over USB, then it
> > becomes a useful alternative to using FEL for loading kernel.
> 
> This is in fact a common use case for development.  Other platforms /
> SoCs do use fastboot in non-Android development as how to pass in a new
> testing kernel / initrd / device tree combo.

AFAIK, there's no support in fastboot for a dt yet. You can only give
it a kernel with an appended DT.

Maxime
Ian Campbell Sept. 7, 2015, 9:07 a.m. UTC | #16
On Sun, 2015-09-06 at 13:22 +0200, Maxime Ripard wrote:
> On Fri, Sep 04, 2015 at 12:59:24PM -0400, Tom Rini wrote:
> > On Tue, Sep 01, 2015 at 09:57:16AM +0200, Maxime Ripard wrote:
> > 
> > [snip]
> > > Of course, this will all depend on the ratio between the empty space
> > > and the files themselves, and what files you actually have there, but
> > > while 32MB is definitely useless, 256MB is already a decent size.
> > 
> > So there's not any automatic chunking and writing of the image?  Maybe
> > I'm confusing this with DFU and/or my wishlist/imagination...
> 
> Apparently, Rob was saying there was some, but I'm not sure we support
> that yet, I'd need to look into it.

It seems this is pretty critical, given that storage (both capacity and
actual required space within the fs) are growing and 256MB is only barely
big enough for the 4GB image you have today (210MB after the various
compression strategies are applied).

Ian.
Rob Herring (Arm) Sept. 8, 2015, 1 p.m. UTC | #17
On Tue, Sep 1, 2015 at 2:14 AM, Maxime Ripard
<maxime.ripard@free-electrons.com> wrote:
> On Mon, Aug 31, 2015 at 02:17:50PM -0500, Rob Herring wrote:
>> On Mon, Aug 31, 2015 at 10:01 AM, Hans de Goede <hdegoede@redhat.com> wrote:
>> > Hi,
>> >
>> > On 31-08-15 16:46, Maxime Ripard wrote:
>> >>
>> >> When using fastboot and flashing a larger image such as the main partition
>> >> of a system, the current 32MB limit for the buffer is quite small.
>> >>
>> >> Increase it to something that looks decent for such a use case.
>> >>
>> >> Signed-off-by: Maxime Ripard <maxime.ripard@free-electrons.com>
>> >> ---
>> >>   include/configs/sunxi-common.h | 2 +-
>> >>   1 file changed, 1 insertion(+), 1 deletion(-)
>> >>
>> >> diff --git a/include/configs/sunxi-common.h
>> >> b/include/configs/sunxi-common.h
>> >> index 1abf73c31179..710521c617f5 100644
>> >> --- a/include/configs/sunxi-common.h
>> >> +++ b/include/configs/sunxi-common.h
>> >> @@ -363,7 +363,7 @@ extern int soft_i2c_gpio_scl;
>> >>   #ifdef CONFIG_USB_FUNCTION_FASTBOOT
>> >>   #define CONFIG_CMD_FASTBOOT
>> >>   #define CONFIG_FASTBOOT_BUF_ADDR      CONFIG_SYS_LOAD_ADDR
>> >> -#define CONFIG_FASTBOOT_BUF_SIZE       0x2000000
>> >> +#define CONFIG_FASTBOOT_BUF_SIZE       (256 << 20)
>> >
>> >
>> > Hmm, where / how does this get allocated? On some boards we only
>> > have 256M RAM, so this is not going to fit ... also if this comes
>> > out of the heap, the current heap is only 4M and the wip sunxi
>> > nand patches boost it to 64 (I still need to verify this works on
>> > a 256M board, this may need a tweak to bootm_size to make sure
>> > the bootm code does not try to put the kernel where it conflicts
>> > with the heap ...).
>>
>> I don't think this needs to be so big with current fastboot tool. It
>> will break up the files if needed. However, IIRC this only works for
>> sparse images, so I think this needs to be sized large enough for your
>> biggest bootimage.
>
> Hmm, interesting.
>
> Do you know how it works exactly ? Are we expected to just go on with
> writing data at the offset we previously stopped in such a case? I
> don't think we support that currently.

The hard work is on the client side. The client will retrieve the
maxdownloadsize variable and then split the sparse image into smaller
hunks if needed. So the u-boot side doesn't have to do anything
special if 2 chunks happen to be contiguous.

Rob
Maxime Ripard Sept. 8, 2015, 3:44 p.m. UTC | #18
On Tue, Sep 08, 2015 at 08:00:50AM -0500, Rob Herring wrote:
> On Tue, Sep 1, 2015 at 2:14 AM, Maxime Ripard
> <maxime.ripard@free-electrons.com> wrote:
> > On Mon, Aug 31, 2015 at 02:17:50PM -0500, Rob Herring wrote:
> >> On Mon, Aug 31, 2015 at 10:01 AM, Hans de Goede <hdegoede@redhat.com> wrote:
> >> > Hi,
> >> >
> >> > On 31-08-15 16:46, Maxime Ripard wrote:
> >> >>
> >> >> When using fastboot and flashing a larger image such as the main partition
> >> >> of a system, the current 32MB limit for the buffer is quite small.
> >> >>
> >> >> Increase it to something that looks decent for such a use case.
> >> >>
> >> >> Signed-off-by: Maxime Ripard <maxime.ripard@free-electrons.com>
> >> >> ---
> >> >>   include/configs/sunxi-common.h | 2 +-
> >> >>   1 file changed, 1 insertion(+), 1 deletion(-)
> >> >>
> >> >> diff --git a/include/configs/sunxi-common.h
> >> >> b/include/configs/sunxi-common.h
> >> >> index 1abf73c31179..710521c617f5 100644
> >> >> --- a/include/configs/sunxi-common.h
> >> >> +++ b/include/configs/sunxi-common.h
> >> >> @@ -363,7 +363,7 @@ extern int soft_i2c_gpio_scl;
> >> >>   #ifdef CONFIG_USB_FUNCTION_FASTBOOT
> >> >>   #define CONFIG_CMD_FASTBOOT
> >> >>   #define CONFIG_FASTBOOT_BUF_ADDR      CONFIG_SYS_LOAD_ADDR
> >> >> -#define CONFIG_FASTBOOT_BUF_SIZE       0x2000000
> >> >> +#define CONFIG_FASTBOOT_BUF_SIZE       (256 << 20)
> >> >
> >> >
> >> > Hmm, where / how does this get allocated? On some boards we only
> >> > have 256M RAM, so this is not going to fit ... also if this comes
> >> > out of the heap, the current heap is only 4M and the wip sunxi
> >> > nand patches boost it to 64 (I still need to verify this works on
> >> > a 256M board, this may need a tweak to bootm_size to make sure
> >> > the bootm code does not try to put the kernel where it conflicts
> >> > with the heap ...).
> >>
> >> I don't think this needs to be so big with current fastboot tool. It
> >> will break up the files if needed. However, IIRC this only works for
> >> sparse images, so I think this needs to be sized large enough for your
> >> biggest bootimage.
> >
> > Hmm, interesting.
> >
> > Do you know how it works exactly ? Are we expected to just go on with
> > writing data at the offset we previously stopped in such a case? I
> > don't think we support that currently.
> 
> The hard work is on the client side. The client will retrieve the
> maxdownloadsize variable and then split the sparse image into smaller
> hunks if needed. So the u-boot side doesn't have to do anything
> special if 2 chunks happen to be contiguous.

Well, it still has to make sure it keeps the offset of the current
session instead of restarting from the partition base offset, and I'm
not sure I've seen anything like it for now.

Maxime
diff mbox

Patch

diff --git a/include/configs/sunxi-common.h b/include/configs/sunxi-common.h
index 1abf73c31179..710521c617f5 100644
--- a/include/configs/sunxi-common.h
+++ b/include/configs/sunxi-common.h
@@ -363,7 +363,7 @@  extern int soft_i2c_gpio_scl;
 #ifdef CONFIG_USB_FUNCTION_FASTBOOT
 #define CONFIG_CMD_FASTBOOT
 #define CONFIG_FASTBOOT_BUF_ADDR	CONFIG_SYS_LOAD_ADDR
-#define CONFIG_FASTBOOT_BUF_SIZE	0x2000000
+#define CONFIG_FASTBOOT_BUF_SIZE	(256 << 20)
 
 #define CONFIG_FASTBOOT_FLASH
 #define CONFIG_FASTBOOT_FLASH_MMC_DEV	0