xen-mapcache: Fix rlimit set size.

Previously, the address space soft limit was set mcache_max_size. So,
before the mcache_max_size was reached by the mapcache, QEMU was killed
for overuse of the virtual address space.

This patch fix that by setting the soft limit to mcache_max_size + 80MB.
I observed that QEMU use 75MB more than max_mcache_size after several
empirical tests.

Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
---
 xen-mapcache.c |   13 ++++++++++---
 1 files changed, 10 insertions(+), 3 deletions(-)

On 08/01/2011 09:26 PM, Anthony PERARD wrote:
> Previously, the address space soft limit was set mcache_max_size. So,
> before the mcache_max_size was reached by the mapcache, QEMU was killed
> for overuse of the virtual address space.
>
> This patch fix that by setting the soft limit to mcache_max_size + 80MB.
> I observed that QEMU use 75MB more than max_mcache_size after several
> empirical tests.

Not sure I understand. What are those 75MB?


Alex

> Signed-off-by: Anthony PERARD<anthony.perard@citrix.com>
> ---
>   xen-mapcache.c |   13 ++++++++++---
>   1 files changed, 10 insertions(+), 3 deletions(-)
>
> diff --git a/xen-mapcache.c b/xen-mapcache.c
> index 007136a..40212f7 100644
> --- a/xen-mapcache.c
> +++ b/xen-mapcache.c
> @@ -40,6 +40,13 @@
>   #endif
>   #define MCACHE_BUCKET_SIZE (1UL<<  MCACHE_BUCKET_SHIFT)
>
> +/* This is the size of the virtual address space reserve to QEMU that will not
> + * be use by MapCache.
> + * From empirical tests I observed that qemu use 75MB more than the
> + * max_mcache_size.
> + */
> +#define NON_MCACHE_MEMORY_SIZE (80 * 1024 * 1024)
> +
>   #define mapcache_lock()   ((void)0)
>   #define mapcache_unlock() ((void)0)
>
> @@ -93,14 +100,14 @@ void xen_map_cache_init(void)
>       mapcache->last_address_index = -1;
>
>       getrlimit(RLIMIT_AS,&rlimit_as);
> -    if (rlimit_as.rlim_max<  MCACHE_MAX_SIZE) {
> +    if (rlimit_as.rlim_max<  MCACHE_MAX_SIZE + NON_MCACHE_MEMORY_SIZE) {
>           rlimit_as.rlim_cur = rlimit_as.rlim_max;
>       } else {
> -        rlimit_as.rlim_cur = MCACHE_MAX_SIZE;
> +        rlimit_as.rlim_cur = MCACHE_MAX_SIZE + NON_MCACHE_MEMORY_SIZE;
>       }
>
>       setrlimit(RLIMIT_AS,&rlimit_as);
> -    mapcache->max_mcache_size = rlimit_as.rlim_cur;
> +    mapcache->max_mcache_size = rlimit_as.rlim_cur - NON_MCACHE_MEMORY_SIZE;
>
>       mapcache->nr_buckets =
>           (((mapcache->max_mcache_size>>  XC_PAGE_SHIFT) +

On Mon, 1 Aug 2011, Anthony PERARD wrote:
> Previously, the address space soft limit was set mcache_max_size. So,
> before the mcache_max_size was reached by the mapcache, QEMU was killed
> for overuse of the virtual address space.
> 
> This patch fix that by setting the soft limit to mcache_max_size + 80MB.
> I observed that QEMU use 75MB more than max_mcache_size after several
> empirical tests.
> 
> Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
> ---
>  xen-mapcache.c |   13 ++++++++++---
>  1 files changed, 10 insertions(+), 3 deletions(-)
> 
> diff --git a/xen-mapcache.c b/xen-mapcache.c
> index 007136a..40212f7 100644
> --- a/xen-mapcache.c
> +++ b/xen-mapcache.c
> @@ -40,6 +40,13 @@
>  #endif
>  #define MCACHE_BUCKET_SIZE (1UL << MCACHE_BUCKET_SHIFT)
>  
> +/* This is the size of the virtual address space reserve to QEMU that will not
> + * be use by MapCache.
> + * From empirical tests I observed that qemu use 75MB more than the
> + * max_mcache_size.
> + */
> +#define NON_MCACHE_MEMORY_SIZE (80 * 1024 * 1024)
> +
>  #define mapcache_lock()   ((void)0)
>  #define mapcache_unlock() ((void)0)
>  
> @@ -93,14 +100,14 @@ void xen_map_cache_init(void)
>      mapcache->last_address_index = -1;
>  
>      getrlimit(RLIMIT_AS, &rlimit_as);
> -    if (rlimit_as.rlim_max < MCACHE_MAX_SIZE) {
> +    if (rlimit_as.rlim_max < MCACHE_MAX_SIZE + NON_MCACHE_MEMORY_SIZE) {
>          rlimit_as.rlim_cur = rlimit_as.rlim_max;
>      } else {
> -        rlimit_as.rlim_cur = MCACHE_MAX_SIZE;
> +        rlimit_as.rlim_cur = MCACHE_MAX_SIZE + NON_MCACHE_MEMORY_SIZE;
>      }

In the 'else' case we should just set rlimit_as.rlim_cur to
rlimit_as.rlim_max.
Also we should detect if we are running as a privileged process and if
that is the case just modify rlimit_as.rlim_max to RLIM_INFINITY.
If we are not running as a privileged process and if rlim_max is not
high enough we should print a warning because trying to guess how much
virtual memory qemu is going to use is obviously error prone.

So this is how I think it should work:

if (rlimit_as.rlim_max < MCACHE_MAX_SIZE) {
    if (qemu is priviledged) {
        rlimit_as.rlim_max = RLIM_INFINITY;
        rlimit_as.rlim_cur = RLIM_INFINITY;
    } else {
        print a warning
        mapcache->max_mcache_size = rlimit_as.rlim_cur - NON_MCACHE_MEMORY_SIZE;
        rlimit_as.rlim_cur = rlimit_as.rlim_max;
    }
} else {
    rlimit_as.rlim_cur = rlimit_as.rlim_max;
}
setrlimit


>      setrlimit(RLIMIT_AS, &rlimit_as);
> -    mapcache->max_mcache_size = rlimit_as.rlim_cur;
> +    mapcache->max_mcache_size = rlimit_as.rlim_cur - NON_MCACHE_MEMORY_SIZE;
>  
>      mapcache->nr_buckets =
>          (((mapcache->max_mcache_size >> XC_PAGE_SHIFT) +
> -- 
> Anthony PERARD
>

On Tue, Aug 2, 2011 at 15:50, Alexander Graf <agraf@suse.de> wrote:
> On 08/01/2011 09:26 PM, Anthony PERARD wrote:
>>
>> Previously, the address space soft limit was set mcache_max_size. So,
>> before the mcache_max_size was reached by the mapcache, QEMU was killed
>> for overuse of the virtual address space.
>>
>> This patch fix that by setting the soft limit to mcache_max_size + 80MB.
>> I observed that QEMU use 75MB more than max_mcache_size after several
>> empirical tests.
>
> Not sure I understand. What are those 75MB?

I did the calculation like that:
 - I make sure that the MapCache was full.
 - I did the difference between the value in mapcache->max_mcache_size
and the size of the used address space by the QEMU process.