| Message ID | 1411459067-840-1-git-send-email-zhang.zhanghailiang@huawei.com |
|---|---|
| State | New |
| Headers | show |
On Tue, Sep 23, 2014 at 03:57:47PM +0800, zhanghailiang wrote: > If we configure mlock=on and memory policy=bind at the same time, > It will consume lots of time for system to treat with memory, > especially when call mbind after mlockall. > > Adjust the place of calling mlockall, calling mbind before mlockall > can remarkably reduce the time of VM's startup. > > Signed-off-by: zhanghailiang <zhang.zhanghailiang@huawei.com> The idea makes absolute sense to me: bind after lock will force data copy of all pages. bind before lock gives us an indication where to put data on fault in. Acked-by: Michael S. Tsirkin <mst@redhat.com> > --- > Hi, > > Actually, for mbind and mlockall, i have made a test about the time consuming > for the different call sequence. > > The results is shown below. It is obviously that mlockall called before mbind is > more time-consuming. > > Besides, this patch is OK with memory hotplug. > > TEST CODE: > if (mbind_first) { > printf("mbind --> mlockall\n"); > mbind(ptr, ram_size/2, MPOL_BIND, &node0mask, 2, > MPOL_MF_STRICT | MPOL_MF_MOVE); > mbind(ptr + ram_size/2, ram_size/2, MPOL_BIND, &node1mask, 2, > MPOL_MF_STRICT | MPOL_MF_MOVE); > mlockall(MCL_CURRENT | MCL_FUTURE); > } else { > printf("mlockall --> mbind\n"); > mlockall(MCL_CURRENT | MCL_FUTURE); > mbind(ptr, ram_size/2, MPOL_BIND, &node0mask, 2 , > MPOL_MF_STRICT | MPOL_MF_MOVE); > mbind(ptr + ram_size/2, ram_size/2, MPOL_BIND, &node1mask, 2, > MPOL_MF_STRICT | MPOL_MF_MOVE); > } > > RESULT 1: > #time /home/test_mbind 10240 0 > memroy size 10737418240 > mlockall --> mbind > > real 0m11.886s > user 0m0.004s > sys 0m11.865s > #time /home/test_mbind 10240 1 > memroy size 10737418240 > mbind --> mlockall > > real 0m5.334s > user 0m0.000s > sys 0m5.324s > > RESULT 2: > #time /home/test_mbind 4096 0 > memroy size 4294967296 > mlockall --> mbind > > real 0m5.503s > user 0m0.000s > sys 0m5.492s > #time /home/test_mbind 4096 1 > memroy size 4294967296 > mbind --> mlockall > > real 0m2.139s > user 0m0.000s > sys 0m2.132s > > Best Regards, > zhanghailiang > --- > vl.c | 11 +++++------ > 1 file changed, 5 insertions(+), 6 deletions(-) > > diff --git a/vl.c b/vl.c > index dc792fe..adf4770 100644 > --- a/vl.c > +++ b/vl.c > @@ -134,6 +134,7 @@ const char* keyboard_layout = NULL; > ram_addr_t ram_size; > const char *mem_path = NULL; > int mem_prealloc = 0; /* force preallocation of physical target memory */ > +int enable_mlock = false; > int nb_nics; > NICInfo nd_table[MAX_NICS]; > int autostart; > @@ -1421,12 +1422,8 @@ static void smp_parse(QemuOpts *opts) > > } > > -static void configure_realtime(QemuOpts *opts) > +static void realtime_init(void) > { > - bool enable_mlock; > - > - enable_mlock = qemu_opt_get_bool(opts, "mlock", true); > - > if (enable_mlock) { > if (os_mlock() < 0) { > fprintf(stderr, "qemu: locking memory failed\n"); > @@ -3973,7 +3970,7 @@ int main(int argc, char **argv, char **envp) > if (!opts) { > exit(1); > } > - configure_realtime(opts); > + enable_mlock = qemu_opt_get_bool(opts, "mlock", true); > break; > case QEMU_OPTION_msg: > opts = qemu_opts_parse(qemu_find_opts("msg"), optarg, 0); > @@ -4441,6 +4438,8 @@ int main(int argc, char **argv, char **envp) > > machine_class->init(current_machine); > > + realtime_init(); > + > audio_init(); > > cpu_synchronize_all_post_init(); > -- > 1.7.12.4 >
On Tue, Sep 23, 2014 at 11:30:26AM +0300, Michael S. Tsirkin wrote: > On Tue, Sep 23, 2014 at 03:57:47PM +0800, zhanghailiang wrote: > > If we configure mlock=on and memory policy=bind at the same time, > > It will consume lots of time for system to treat with memory, > > especially when call mbind after mlockall. > > > > Adjust the place of calling mlockall, calling mbind before mlockall > > can remarkably reduce the time of VM's startup. > > > > Signed-off-by: zhanghailiang <zhang.zhanghailiang@huawei.com> > > The idea makes absolute sense to me: > bind after lock will force data copy of > all pages. bind before lock gives us an > indication where to put data on fault in. Agreed. > > Acked-by: Michael S. Tsirkin <mst@redhat.com> > > > > --- > > Hi, > > > > Actually, for mbind and mlockall, i have made a test about the time consuming > > for the different call sequence. > > > > The results is shown below. It is obviously that mlockall called before mbind is > > more time-consuming. > > > > Besides, this patch is OK with memory hotplug. > > > > TEST CODE: > > if (mbind_first) { > > printf("mbind --> mlockall\n"); > > mbind(ptr, ram_size/2, MPOL_BIND, &node0mask, 2, > > MPOL_MF_STRICT | MPOL_MF_MOVE); > > mbind(ptr + ram_size/2, ram_size/2, MPOL_BIND, &node1mask, 2, > > MPOL_MF_STRICT | MPOL_MF_MOVE); > > mlockall(MCL_CURRENT | MCL_FUTURE); > > } else { > > printf("mlockall --> mbind\n"); > > mlockall(MCL_CURRENT | MCL_FUTURE); > > mbind(ptr, ram_size/2, MPOL_BIND, &node0mask, 2 , > > MPOL_MF_STRICT | MPOL_MF_MOVE); > > mbind(ptr + ram_size/2, ram_size/2, MPOL_BIND, &node1mask, 2, > > MPOL_MF_STRICT | MPOL_MF_MOVE); > > } > > > > RESULT 1: > > #time /home/test_mbind 10240 0 > > memroy size 10737418240 > > mlockall --> mbind > > > > real 0m11.886s > > user 0m0.004s > > sys 0m11.865s > > #time /home/test_mbind 10240 1 > > memroy size 10737418240 > > mbind --> mlockall > > > > real 0m5.334s > > user 0m0.000s > > sys 0m5.324s > > > > RESULT 2: > > #time /home/test_mbind 4096 0 > > memroy size 4294967296 > > mlockall --> mbind > > > > real 0m5.503s > > user 0m0.000s > > sys 0m5.492s > > #time /home/test_mbind 4096 1 > > memroy size 4294967296 > > mbind --> mlockall > > > > real 0m2.139s > > user 0m0.000s > > sys 0m2.132s > > > > Best Regards, > > zhanghailiang > > --- > > vl.c | 11 +++++------ > > 1 file changed, 5 insertions(+), 6 deletions(-) > > > > diff --git a/vl.c b/vl.c > > index dc792fe..adf4770 100644 > > --- a/vl.c > > +++ b/vl.c > > @@ -134,6 +134,7 @@ const char* keyboard_layout = NULL; > > ram_addr_t ram_size; > > const char *mem_path = NULL; > > int mem_prealloc = 0; /* force preallocation of physical target memory */ > > +int enable_mlock = false; Why not bool? Regards, Hu
On 2014/9/23 16:35, Hu Tao wrote: > On Tue, Sep 23, 2014 at 11:30:26AM +0300, Michael S. Tsirkin wrote: >> On Tue, Sep 23, 2014 at 03:57:47PM +0800, zhanghailiang wrote: >>> If we configure mlock=on and memory policy=bind at the same time, >>> It will consume lots of time for system to treat with memory, >>> especially when call mbind after mlockall. >>> >>> Adjust the place of calling mlockall, calling mbind before mlockall >>> can remarkably reduce the time of VM's startup. >>> >>> Signed-off-by: zhanghailiang <zhang.zhanghailiang@huawei.com> >> >> The idea makes absolute sense to me: >> bind after lock will force data copy of >> all pages. bind before lock gives us an >> indication where to put data on fault in. > > Agreed. > >> >> Acked-by: Michael S. Tsirkin <mst@redhat.com> >> >> >>> --- >>> Hi, >>> >>> Actually, for mbind and mlockall, i have made a test about the time consuming >>> for the different call sequence. >>> >>> The results is shown below. It is obviously that mlockall called before mbind is >>> more time-consuming. >>> >>> Besides, this patch is OK with memory hotplug. >>> >>> TEST CODE: >>> if (mbind_first) { >>> printf("mbind --> mlockall\n"); >>> mbind(ptr, ram_size/2, MPOL_BIND, &node0mask, 2, >>> MPOL_MF_STRICT | MPOL_MF_MOVE); >>> mbind(ptr + ram_size/2, ram_size/2, MPOL_BIND, &node1mask, 2, >>> MPOL_MF_STRICT | MPOL_MF_MOVE); >>> mlockall(MCL_CURRENT | MCL_FUTURE); >>> } else { >>> printf("mlockall --> mbind\n"); >>> mlockall(MCL_CURRENT | MCL_FUTURE); >>> mbind(ptr, ram_size/2, MPOL_BIND, &node0mask, 2 , >>> MPOL_MF_STRICT | MPOL_MF_MOVE); >>> mbind(ptr + ram_size/2, ram_size/2, MPOL_BIND, &node1mask, 2, >>> MPOL_MF_STRICT | MPOL_MF_MOVE); >>> } >>> >>> RESULT 1: >>> #time /home/test_mbind 10240 0 >>> memroy size 10737418240 >>> mlockall --> mbind >>> >>> real 0m11.886s >>> user 0m0.004s >>> sys 0m11.865s >>> #time /home/test_mbind 10240 1 >>> memroy size 10737418240 >>> mbind --> mlockall >>> >>> real 0m5.334s >>> user 0m0.000s >>> sys 0m5.324s >>> >>> RESULT 2: >>> #time /home/test_mbind 4096 0 >>> memroy size 4294967296 >>> mlockall --> mbind >>> >>> real 0m5.503s >>> user 0m0.000s >>> sys 0m5.492s >>> #time /home/test_mbind 4096 1 >>> memroy size 4294967296 >>> mbind --> mlockall >>> >>> real 0m2.139s >>> user 0m0.000s >>> sys 0m2.132s >>> >>> Best Regards, >>> zhanghailiang >>> --- >>> vl.c | 11 +++++------ >>> 1 file changed, 5 insertions(+), 6 deletions(-) >>> >>> diff --git a/vl.c b/vl.c >>> index dc792fe..adf4770 100644 >>> --- a/vl.c >>> +++ b/vl.c >>> @@ -134,6 +134,7 @@ const char* keyboard_layout = NULL; >>> ram_addr_t ram_size; >>> const char *mem_path = NULL; >>> int mem_prealloc = 0; /* force preallocation of physical target memory */ >>> +int enable_mlock = false; > > Why not bool? > Er, that is my fault, Will fix it and submit V2, Thanks;) > Regards, > Hu > > . >
On 2014/9/23 16:30, Michael S. Tsirkin wrote: > On Tue, Sep 23, 2014 at 03:57:47PM +0800, zhanghailiang wrote: >> If we configure mlock=on and memory policy=bind at the same time, >> It will consume lots of time for system to treat with memory, >> especially when call mbind after mlockall. >> >> Adjust the place of calling mlockall, calling mbind before mlockall >> can remarkably reduce the time of VM's startup. >> >> Signed-off-by: zhanghailiang <zhang.zhanghailiang@huawei.com> > > The idea makes absolute sense to me: > bind after lock will force data copy of > all pages. bind before lock gives us an > indication where to put data on fault in. > > Acked-by: Michael S. Tsirkin <mst@redhat.com> > > Thanks for your quick reviewing.. Best Regards, zhanghailiang >> --- >> Hi, >> >> Actually, for mbind and mlockall, i have made a test about the time consuming >> for the different call sequence. >> >> The results is shown below. It is obviously that mlockall called before mbind is >> more time-consuming. >> >> Besides, this patch is OK with memory hotplug. >> >> TEST CODE: >> if (mbind_first) { >> printf("mbind --> mlockall\n"); >> mbind(ptr, ram_size/2, MPOL_BIND, &node0mask, 2, >> MPOL_MF_STRICT | MPOL_MF_MOVE); >> mbind(ptr + ram_size/2, ram_size/2, MPOL_BIND, &node1mask, 2, >> MPOL_MF_STRICT | MPOL_MF_MOVE); >> mlockall(MCL_CURRENT | MCL_FUTURE); >> } else { >> printf("mlockall --> mbind\n"); >> mlockall(MCL_CURRENT | MCL_FUTURE); >> mbind(ptr, ram_size/2, MPOL_BIND, &node0mask, 2 , >> MPOL_MF_STRICT | MPOL_MF_MOVE); >> mbind(ptr + ram_size/2, ram_size/2, MPOL_BIND, &node1mask, 2, >> MPOL_MF_STRICT | MPOL_MF_MOVE); >> } >> >> RESULT 1: >> #time /home/test_mbind 10240 0 >> memroy size 10737418240 >> mlockall --> mbind >> >> real 0m11.886s >> user 0m0.004s >> sys 0m11.865s >> #time /home/test_mbind 10240 1 >> memroy size 10737418240 >> mbind --> mlockall >> >> real 0m5.334s >> user 0m0.000s >> sys 0m5.324s >> >> RESULT 2: >> #time /home/test_mbind 4096 0 >> memroy size 4294967296 >> mlockall --> mbind >> >> real 0m5.503s >> user 0m0.000s >> sys 0m5.492s >> #time /home/test_mbind 4096 1 >> memroy size 4294967296 >> mbind --> mlockall >> >> real 0m2.139s >> user 0m0.000s >> sys 0m2.132s >> >> Best Regards, >> zhanghailiang >> --- >> vl.c | 11 +++++------ >> 1 file changed, 5 insertions(+), 6 deletions(-) >> >> diff --git a/vl.c b/vl.c >> index dc792fe..adf4770 100644 >> --- a/vl.c >> +++ b/vl.c >> @@ -134,6 +134,7 @@ const char* keyboard_layout = NULL; >> ram_addr_t ram_size; >> const char *mem_path = NULL; >> int mem_prealloc = 0; /* force preallocation of physical target memory */ >> +int enable_mlock = false; >> int nb_nics; >> NICInfo nd_table[MAX_NICS]; >> int autostart; >> @@ -1421,12 +1422,8 @@ static void smp_parse(QemuOpts *opts) >> >> } >> >> -static void configure_realtime(QemuOpts *opts) >> +static void realtime_init(void) >> { >> - bool enable_mlock; >> - >> - enable_mlock = qemu_opt_get_bool(opts, "mlock", true); >> - >> if (enable_mlock) { >> if (os_mlock() < 0) { >> fprintf(stderr, "qemu: locking memory failed\n"); >> @@ -3973,7 +3970,7 @@ int main(int argc, char **argv, char **envp) >> if (!opts) { >> exit(1); >> } >> - configure_realtime(opts); >> + enable_mlock = qemu_opt_get_bool(opts, "mlock", true); >> break; >> case QEMU_OPTION_msg: >> opts = qemu_opts_parse(qemu_find_opts("msg"), optarg, 0); >> @@ -4441,6 +4438,8 @@ int main(int argc, char **argv, char **envp) >> >> machine_class->init(current_machine); >> >> + realtime_init(); >> + >> audio_init(); >> >> cpu_synchronize_all_post_init(); >> -- >> 1.7.12.4 >> > > . >
diff --git a/vl.c b/vl.c index dc792fe..adf4770 100644 --- a/vl.c +++ b/vl.c @@ -134,6 +134,7 @@ const char* keyboard_layout = NULL; ram_addr_t ram_size; const char *mem_path = NULL; int mem_prealloc = 0; /* force preallocation of physical target memory */ +int enable_mlock = false; int nb_nics; NICInfo nd_table[MAX_NICS]; int autostart; @@ -1421,12 +1422,8 @@ static void smp_parse(QemuOpts *opts) } -static void configure_realtime(QemuOpts *opts) +static void realtime_init(void) { - bool enable_mlock; - - enable_mlock = qemu_opt_get_bool(opts, "mlock", true); - if (enable_mlock) { if (os_mlock() < 0) { fprintf(stderr, "qemu: locking memory failed\n"); @@ -3973,7 +3970,7 @@ int main(int argc, char **argv, char **envp) if (!opts) { exit(1); } - configure_realtime(opts); + enable_mlock = qemu_opt_get_bool(opts, "mlock", true); break; case QEMU_OPTION_msg: opts = qemu_opts_parse(qemu_find_opts("msg"), optarg, 0); @@ -4441,6 +4438,8 @@ int main(int argc, char **argv, char **envp) machine_class->init(current_machine); + realtime_init(); + audio_init(); cpu_synchronize_all_post_init();
If we configure mlock=on and memory policy=bind at the same time, It will consume lots of time for system to treat with memory, especially when call mbind after mlockall. Adjust the place of calling mlockall, calling mbind before mlockall can remarkably reduce the time of VM's startup. Signed-off-by: zhanghailiang <zhang.zhanghailiang@huawei.com> --- Hi, Actually, for mbind and mlockall, i have made a test about the time consuming for the different call sequence. The results is shown below. It is obviously that mlockall called before mbind is more time-consuming. Besides, this patch is OK with memory hotplug. TEST CODE: if (mbind_first) { printf("mbind --> mlockall\n"); mbind(ptr, ram_size/2, MPOL_BIND, &node0mask, 2, MPOL_MF_STRICT | MPOL_MF_MOVE); mbind(ptr + ram_size/2, ram_size/2, MPOL_BIND, &node1mask, 2, MPOL_MF_STRICT | MPOL_MF_MOVE); mlockall(MCL_CURRENT | MCL_FUTURE); } else { printf("mlockall --> mbind\n"); mlockall(MCL_CURRENT | MCL_FUTURE); mbind(ptr, ram_size/2, MPOL_BIND, &node0mask, 2 , MPOL_MF_STRICT | MPOL_MF_MOVE); mbind(ptr + ram_size/2, ram_size/2, MPOL_BIND, &node1mask, 2, MPOL_MF_STRICT | MPOL_MF_MOVE); } RESULT 1: #time /home/test_mbind 10240 0 memroy size 10737418240 mlockall --> mbind real 0m11.886s user 0m0.004s sys 0m11.865s #time /home/test_mbind 10240 1 memroy size 10737418240 mbind --> mlockall real 0m5.334s user 0m0.000s sys 0m5.324s RESULT 2: #time /home/test_mbind 4096 0 memroy size 4294967296 mlockall --> mbind real 0m5.503s user 0m0.000s sys 0m5.492s #time /home/test_mbind 4096 1 memroy size 4294967296 mbind --> mlockall real 0m2.139s user 0m0.000s sys 0m2.132s Best Regards, zhanghailiang --- vl.c | 11 +++++------ 1 file changed, 5 insertions(+), 6 deletions(-)