| Message ID | 20200522141937.3523692-4-heiko@sntech.de |
|---|---|
| State | Superseded |
| Delegated to: | Tom Rini |
| Headers | show |
| Series | [v5,1/8] lib: rsa: distinguish between tpl and spl for CONFIG_RSA_VERIFY | expand |
Am Freitag, 22. Mai 2020, 16:19:33 CEST schrieb Heiko Stuebner: > From: Heiko Stuebner <heiko.stuebner@theobroma-systems.com> > > When calculating rrtmp/rr rsa_gen_key_prop() tries to make > (((rlen + 31) >> 5) + 1) steps in the rr uint32_t array and > (((rlen + 7) >> 3) + 1) / 4 steps in uint32_t rrtmp[] > with rlen being num_bits * 2 > > On a 4096bit key this comes down to to 257 uint32_t elements > in rr and 256 elements in rrtmp but with the current allocation > rr and rrtmp only have 129 uint32_t elements. > > On 2048bit keys this works by chance as the defined max_rsa_size=4096 > allocates a suitable number of elements, but with an actual 4096bit key > this results in other memory parts getting overwritten. > > so double the number of elements in rr and rrtmp so that it matches > the needed number and should increase nicely if max_rsa_size gets > increased in the future. > > Signed-off-by: Heiko Stuebner <heiko.stuebner@theobroma-systems.com> transplanting a tag from v4: Reviewed-by: Simon Glass <sjg@chromium.org>
> On 22.05.2020, at 16:19, Heiko Stuebner <heiko@sntech.de> wrote: > > From: Heiko Stuebner <heiko.stuebner@theobroma-systems.com> > > When calculating rrtmp/rr rsa_gen_key_prop() tries to make > (((rlen + 31) >> 5) + 1) steps in the rr uint32_t array and > (((rlen + 7) >> 3) + 1) / 4 steps in uint32_t rrtmp[] > with rlen being num_bits * 2 > > On a 4096bit key this comes down to to 257 uint32_t elements > in rr and 256 elements in rrtmp but with the current allocation > rr and rrtmp only have 129 uint32_t elements. > > On 2048bit keys this works by chance as the defined max_rsa_size=4096 > allocates a suitable number of elements, but with an actual 4096bit key > this results in other memory parts getting overwritten. > > so double the number of elements in rr and rrtmp so that it matches > the needed number and should increase nicely if max_rsa_size gets > increased in the future. > > Signed-off-by: Heiko Stuebner <heiko.stuebner@theobroma-systems.com> Reviewed-by: Philipp Tomsich <philipp.tomsich@theobroma-systems.com>
On 25.05.20 22:43, Philipp Tomsich wrote: > > >> On 22.05.2020, at 16:19, Heiko Stuebner <heiko@sntech.de> wrote: >> >> From: Heiko Stuebner <heiko.stuebner@theobroma-systems.com> >> >> When calculating rrtmp/rr rsa_gen_key_prop() tries to make >> (((rlen + 31) >> 5) + 1) steps in the rr uint32_t array and >> (((rlen + 7) >> 3) + 1) / 4 steps in uint32_t rrtmp[] >> with rlen being num_bits * 2 >> >> On a 4096bit key this comes down to to 257 uint32_t elements >> in rr and 256 elements in rrtmp but with the current allocation >> rr and rrtmp only have 129 uint32_t elements. >> >> On 2048bit keys this works by chance as the defined max_rsa_size=4096 >> allocates a suitable number of elements, but with an actual 4096bit key >> this results in other memory parts getting overwritten. >> >> so double the number of elements in rr and rrtmp so that it matches >> the needed number and should increase nicely if max_rsa_size gets >> increased in the future. >> >> Signed-off-by: Heiko Stuebner <heiko.stuebner@theobroma-systems.com> > > Reviewed-by: Philipp Tomsich <philipp.tomsich@theobroma-systems.com> > Why would we allocate memory according to max_rsa_size and not according to the actual size (*prop)->num_bits = (rsa_key.n_sz - i) * 8; Who stops a user from using 8192 or 16384 bits? We should avoid any constant here if we do not validate that it is not exceeded. openssl genrsa -out mykey.pem 8192 works fine for me. Best regards Heinrich
diff --git a/lib/rsa/rsa-keyprop.c b/lib/rsa/rsa-keyprop.c index 4b54db44c4..e28fbb7472 100644 --- a/lib/rsa/rsa-keyprop.c +++ b/lib/rsa/rsa-keyprop.c @@ -659,8 +659,8 @@ int rsa_gen_key_prop(const void *key, uint32_t keylen, struct key_prop **prop) *prop = calloc(sizeof(**prop), 1); n = calloc(sizeof(uint32_t), 1 + (max_rsa_size >> 5)); - rr = calloc(sizeof(uint32_t), 1 + (max_rsa_size >> 5)); - rrtmp = calloc(sizeof(uint32_t), 1 + (max_rsa_size >> 5)); + rr = calloc(sizeof(uint32_t), 1 + ((max_rsa_size * 2) >> 5)); + rrtmp = calloc(sizeof(uint32_t), 1 + ((max_rsa_size * 2) >> 5)); if (!(*prop) || !n || !rr || !rrtmp) { ret = -ENOMEM; goto err;