From patchwork Thu Jun 28 02:54:58 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Stewart Smith X-Patchwork-Id: 935861 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) (using TLSv1.2 with cipher ADH-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 41GPXr0yMRz9ry1 for ; Thu, 28 Jun 2018 12:55:40 +1000 (AEST) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com Received: from lists.ozlabs.org (lists.ozlabs.org [IPv6:2401:3900:2:1::3]) by lists.ozlabs.org (Postfix) with ESMTP id 41GPXq6MLtzF1Kr for ; Thu, 28 Jun 2018 12:55:39 +1000 (AEST) Authentication-Results: lists.ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com X-Original-To: skiboot@lists.ozlabs.org Delivered-To: skiboot@lists.ozlabs.org Authentication-Results: lists.ozlabs.org; spf=pass (mailfrom) smtp.mailfrom=linux.ibm.com (client-ip=148.163.156.1; helo=mx0a-001b2d01.pphosted.com; envelope-from=stewart@linux.ibm.com; receiver=) Authentication-Results: lists.ozlabs.org; dmarc=none (p=none dis=none) header.from=linux.ibm.com Received: from mx0a-001b2d01.pphosted.com (mx0a-001b2d01.pphosted.com [148.163.156.1]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by lists.ozlabs.org (Postfix) with ESMTPS id 41GPXj3qbZzF1JL for ; Thu, 28 Jun 2018 12:55:32 +1000 (AEST) Received: from pps.filterd (m0098399.ppops.net [127.0.0.1]) by mx0a-001b2d01.pphosted.com (8.16.0.22/8.16.0.22) with SMTP id w5S2s7ns139546 for ; Wed, 27 Jun 2018 22:55:30 -0400 Received: from e34.co.us.ibm.com (e34.co.us.ibm.com [32.97.110.152]) by mx0a-001b2d01.pphosted.com with ESMTP id 2jvmsr52vc-1 (version=TLSv1.2 cipher=AES256-GCM-SHA384 bits=256 verify=NOT) for ; Wed, 27 Jun 2018 22:55:30 -0400 Received: from localhost by e34.co.us.ibm.com with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted for from ; Wed, 27 Jun 2018 20:55:29 -0600 Received: from b03cxnp08026.gho.boulder.ibm.com (9.17.130.18) by e34.co.us.ibm.com (192.168.1.134) with IBM ESMTP SMTP Gateway: Authorized Use Only! Violators will be prosecuted; (version=TLSv1/SSLv3 cipher=AES256-GCM-SHA384 bits=256/256) Wed, 27 Jun 2018 20:55:27 -0600 Received: from b03ledav005.gho.boulder.ibm.com (b03ledav005.gho.boulder.ibm.com [9.17.130.236]) by b03cxnp08026.gho.boulder.ibm.com (8.14.9/8.14.9/NCO v10.0) with ESMTP id w5S2tQKk11141612 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-GCM-SHA384 bits=256 verify=FAIL); Wed, 27 Jun 2018 19:55:27 -0700 Received: from b03ledav005.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id D9176BE04F; Wed, 27 Jun 2018 20:55:26 -0600 (MDT) Received: from b03ledav005.gho.boulder.ibm.com (unknown [127.0.0.1]) by IMSVA (Postfix) with ESMTP id 03742BE056; Wed, 27 Jun 2018 20:55:25 -0600 (MDT) Received: from birb.localdomain (unknown [9.102.42.25]) by b03ledav005.gho.boulder.ibm.com (Postfix) with SMTP; Wed, 27 Jun 2018 20:55:25 -0600 (MDT) Received: by birb.localdomain (Postfix, from userid 1000) id 9C1474EC6A1; Thu, 28 Jun 2018 12:55:21 +1000 (AEST) From: Stewart Smith To: Nicholas Piggin , Stewart Smith Date: Thu, 28 Jun 2018 12:54:58 +1000 X-Mailer: git-send-email 2.17.1 In-Reply-To: <20180628025501.20676-1-stewart@linux.ibm.com> References: <20180418135513.64817b17@roar.ozlabs.ibm.com> <20180628025501.20676-1-stewart@linux.ibm.com> In-Reply-To: <20180418135513.64817b17@roar.ozlabs.ibm.com> References: <20180418135513.64817b17@roar.ozlabs.ibm.com> X-TM-AS-GCONF: 00 x-cbid: 18062802-0016-0000-0000-0000090055F5 X-IBM-SpamModules-Scores: X-IBM-SpamModules-Versions: BY=3.00009267; HX=3.00000241; KW=3.00000007; PH=3.00000004; SC=3.00000266; SDB=6.01053344; UDB=6.00540074; IPR=6.00831268; MB=3.00021894; MTD=3.00000008; XFM=3.00000015; UTC=2018-06-28 02:55:28 X-IBM-AV-DETECTION: SAVI=unused REMOTE=unused XFE=unused x-cbparentid: 18062802-0017-0000-0000-00003F6F645A Message-Id: <20180628025501.20676-4-stewart@linux.ibm.com> X-Proofpoint-Virus-Version: vendor=fsecure engine=2.50.10434:, , definitions=2018-06-27_08:, , signatures=0 X-Proofpoint-Spam-Details: rule=outbound_notspam policy=outbound score=0 priorityscore=1501 malwarescore=0 suspectscore=2 phishscore=0 bulkscore=0 spamscore=0 clxscore=1015 lowpriorityscore=0 mlxscore=0 impostorscore=0 mlxlogscore=999 adultscore=0 classifier=spam adjust=0 reason=mlx scancount=1 engine=8.0.1-1806210000 definitions=main-1806280029 Subject: [Skiboot] [RFC v2 PATCH 3/6] Scan PCI and clear memory simultaneously X-BeenThere: skiboot@lists.ozlabs.org X-Mailman-Version: 2.1.26 Precedence: list List-Id: Mailing list for skiboot development List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Cc: skiboot@lists.ozlabs.org MIME-Version: 1.0 Errors-To: skiboot-bounces+incoming=patchwork.ozlabs.org@lists.ozlabs.org Sender: "Skiboot" For many systems, scanning PCI takes about as much time as zeroing all of RAM, so we may as well do them at the same time and cut a few seconds off the total fast reboot time. Signed-off-by: Stewart Smith --- core/fast-reboot.c | 5 ++++- core/mem_region.c | 51 +++++++++++++++++++++++++++++--------------- include/mem_region.h | 3 ++- 3 files changed, 40 insertions(+), 19 deletions(-) diff --git a/core/fast-reboot.c b/core/fast-reboot.c index 8234799bb7a7..b7e518b1a492 100644 --- a/core/fast-reboot.c +++ b/core/fast-reboot.c @@ -371,6 +371,9 @@ void __noreturn fast_reboot_entry(void) /* Start preloading kernel and ramdisk */ start_preload_kernel(); + /* Start clearing memory */ + start_mem_region_clear_unused(); + /* Poke the consoles (see comments in the code there) */ fsp_console_reset(); @@ -396,7 +399,7 @@ void __noreturn fast_reboot_entry(void) ipmi_set_fw_progress_sensor(IPMI_FW_PCI_INIT); - mem_region_clear_unused(); + wait_mem_region_clear_unused(); /* Load and boot payload */ load_and_boot_kernel(true); diff --git a/core/mem_region.c b/core/mem_region.c index a8f18b01a17b..b8ed15fba74d 100644 --- a/core/mem_region.c +++ b/core/mem_region.c @@ -1224,31 +1224,38 @@ static void mem_region_clear_job(void *data) #define MEM_REGION_CLEAR_JOB_SIZE (16ULL*(1<<30)) -void mem_region_clear_unused(void) +static struct cpu_job **mem_clear_jobs; +static struct mem_region_clear_job_args *mem_clear_job_args; +static int mem_clear_njobs = 0; + +void start_mem_region_clear_unused(void) { - int njobs = 0; - struct cpu_job **jobs; struct mem_region *r; - struct mem_region_clear_job_args *job_args; uint64_t s,l; uint64_t total = 0; uint32_t chip_id; char *path; int i; + struct cpu_job **jobs; + struct mem_region_clear_job_args *job_args; lock(&mem_region_lock); assert(mem_regions_finalised); + mem_clear_njobs = 0; + list_for_each(®ions, r, list) { if (!(r->type == REGION_OS)) continue; - njobs++; + mem_clear_njobs++; /* One job per 16GB */ - njobs += r->len / MEM_REGION_CLEAR_JOB_SIZE; + mem_clear_njobs += r->len / MEM_REGION_CLEAR_JOB_SIZE; } - jobs = malloc(njobs * sizeof(struct cpu_job*)); - job_args = malloc(njobs * sizeof(struct mem_region_clear_job_args)); + jobs = malloc(mem_clear_njobs * sizeof(struct cpu_job*)); + job_args = malloc(mem_clear_njobs * sizeof(struct mem_region_clear_job_args)); + mem_clear_jobs = jobs; + mem_clear_job_args = job_args; prlog(PR_NOTICE, "Clearing unused memory:\n"); i = 0; @@ -1278,7 +1285,6 @@ void mem_region_clear_unused(void) (job_args[i].e - job_args[i].s), chip_id); free(path); - printf("job: %s\n", job_args[i].job_name); jobs[i] = cpu_queue_job_on_node(chip_id, job_args[i].job_name, mem_region_clear_job, @@ -1300,25 +1306,36 @@ void mem_region_clear_unused(void) (job_args[i].e - job_args[i].s), chip_id); free(path); - printf("job: %s\n", job_args[i].job_name); jobs[i] = cpu_queue_job_on_node(chip_id, job_args[i].job_name, mem_region_clear_job, &job_args[i]); i++; } + unlock(&mem_region_lock); cpu_process_local_jobs(); +} + +void wait_mem_region_clear_unused(void) +{ + uint64_t l; + uint64_t total = 0; + int i; + + for(i=0; i < mem_clear_njobs; i++) { + total += (mem_clear_job_args[i].e - mem_clear_job_args[i].s); + } + l = 0; - for(i=0; i < njobs; i++) { - cpu_wait_job(jobs[i], true); - l += (job_args[i].e - job_args[i].s); + for(i=0; i < mem_clear_njobs; i++) { + cpu_wait_job(mem_clear_jobs[i], true); + l += (mem_clear_job_args[i].e - mem_clear_job_args[i].s); printf("Clearing memory... %"PRIu64"/%"PRIu64"GB done\n", l>>30, total>>30); - free(job_args[i].job_name); + free(mem_clear_job_args[i].job_name); } - unlock(&mem_region_lock); - free(jobs); - free(job_args); + free(mem_clear_jobs); + free(mem_clear_job_args); } static void mem_region_add_dt_reserved_node(struct dt_node *parent, diff --git a/include/mem_region.h b/include/mem_region.h index 9e288dac45be..018dfa0e5eb0 100644 --- a/include/mem_region.h +++ b/include/mem_region.h @@ -65,7 +65,8 @@ size_t mem_allocated_size(const void *ptr); bool mem_check(const struct mem_region *region); bool mem_check_all(void); void mem_region_release_unused(void); -void mem_region_clear_unused(void); +void start_mem_region_clear_unused(void); +void wait_mem_region_clear_unused(void); int64_t mem_dump_free(void); void mem_dump_allocs(void);