From patchwork Mon Nov 21 23:40:44 2011 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Andy Fleming X-Patchwork-Id: 126968 X-Patchwork-Delegate: wd@denx.de Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from theia.denx.de (theia.denx.de [85.214.87.163]) by ozlabs.org (Postfix) with ESMTP id 5C1A8B71E7 for ; Tue, 22 Nov 2011 10:41:10 +1100 (EST) Received: from localhost (localhost [127.0.0.1]) by theia.denx.de (Postfix) with ESMTP id 93DCD28426; Tue, 22 Nov 2011 00:41:08 +0100 (CET) X-Virus-Scanned: Debian amavisd-new at theia.denx.de Received: from theia.denx.de ([127.0.0.1]) by localhost (theia.denx.de [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id f7zn8HTNusBx; Tue, 22 Nov 2011 00:41:08 +0100 (CET) Received: from theia.denx.de (localhost [127.0.0.1]) by theia.denx.de (Postfix) with ESMTP id 81F522842A; Tue, 22 Nov 2011 00:41:06 +0100 (CET) Received: from localhost (localhost [127.0.0.1]) by theia.denx.de (Postfix) with ESMTP id E0D6428417 for ; Tue, 22 Nov 2011 00:41:03 +0100 (CET) X-Virus-Scanned: Debian amavisd-new at theia.denx.de Received: from theia.denx.de ([127.0.0.1]) by localhost (theia.denx.de [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id Ku+6lz+YF49w for ; Tue, 22 Nov 2011 00:41:03 +0100 (CET) X-policyd-weight: NOT_IN_SBL_XBL_SPAMHAUS=-1.5 NOT_IN_SPAMCOP=-1.5 NOT_IN_BL_NJABL=-1.5 (only DNSBL check requested) Received: from VA3EHSOBE010.bigfish.com (mail-va3.bigfish.com [216.32.180.10]) by theia.denx.de (Postfix) with ESMTPS id BDB3F28426 for ; Tue, 22 Nov 2011 00:40:57 +0100 (CET) Received: from mail139-va3-R.bigfish.com (10.7.14.252) by VA3EHSOBE010.bigfish.com (10.7.40.12) with Microsoft SMTP Server id 14.1.225.22; Mon, 21 Nov 2011 23:40:10 +0000 Received: from mail139-va3 (localhost [127.0.0.1]) by mail139-va3-R.bigfish.com (Postfix) with ESMTP id 317B040298; Mon, 21 Nov 2011 23:40:07 +0000 (UTC) X-SpamScore: 0 X-BigFish: VS0(zzzz1202hzz8275bhz2dh2a8h668h839h61h) X-Spam-TCS-SCL: 0:0 X-Forefront-Antispam-Report: CIP:70.37.183.190; KIP:(null); UIP:(null); IPV:NLI; H:mail.freescale.net; RD:none; EFVD:NLI Received: from mail139-va3 (localhost.localdomain [127.0.0.1]) by mail139-va3 (MessageSwitch) id 1321918806816683_24971; Mon, 21 Nov 2011 23:40:06 +0000 (UTC) Received: from VA3EHSMHS028.bigfish.com (unknown [10.7.14.245]) by mail139-va3.bigfish.com (Postfix) with ESMTP id BFF9D700045; Mon, 21 Nov 2011 23:40:06 +0000 (UTC) Received: from mail.freescale.net (70.37.183.190) by VA3EHSMHS028.bigfish.com (10.7.99.38) with Microsoft SMTP Server (TLS) id 14.1.225.22; Mon, 21 Nov 2011 23:40:08 +0000 Received: from az33smr02.freescale.net (10.64.34.200) by 039-SN1MMR1-003.039d.mgd.msft.net (10.84.1.16) with Microsoft SMTP Server id 14.1.339.2; Mon, 21 Nov 2011 17:40:46 -0600 Received: from localhost (right.am.freescale.net [10.82.193.13]) by az33smr02.freescale.net (8.13.1/8.13.0) with ESMTP id pALNejX5011911; Mon, 21 Nov 2011 17:40:46 -0600 (CST) From: Andy Fleming To: Wolfgang Denk Date: Mon, 21 Nov 2011 17:40:44 -0600 Message-ID: <1321918844-19597-2-git-send-email-afleming@freescale.com> X-Mailer: git-send-email 1.7.3.4 In-Reply-To: <1321918844-19597-1-git-send-email-afleming@freescale.com> References: <1321918844-19597-1-git-send-email-afleming@freescale.com> MIME-Version: 1.0 X-OriginatorOrg: freescale.com Cc: u-boot@lists.denx.de Subject: [U-Boot] [PATCH 2/2 v2] Allow for parallel builds and saved output X-BeenThere: u-boot@lists.denx.de X-Mailman-Version: 2.1.11 Precedence: list List-Id: U-Boot discussion List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: u-boot-bounces@lists.denx.de Errors-To: u-boot-bounces@lists.denx.de The MAKEALL script cleverly runs make with the appropriate options to use all of the cores on the system, but your average U-Boot build can't make much use of more than a few cores. If you happen to have a many-core server, your builds will leave most of the system idle. In order to make full use of such a system, we need to build multiple targets in parallel, and this requires directing make output into multiple directories. We add a BUILD_NBUILDS variable, which allows users to specify how many builds to run in parallel. When BUILD_NBUILDS is set greater than 1, we redefine BUILD_DIR for each build to be ${BUILD_DIR}/${target}. Also, we make "./build" the default BUILD_DIR when BUILD_NBUILDS is greater than 1. MAKEALL now tracks which builds are still running, and when one finishes, it starts a new build. Once each build finishes, we run "make tidy" on its directory, to reduce the footprint. As a result, we are left with a build directory with all of the built targets still there for use, which means anyone who wanted to use MAKEALL as part of a test harness can now do so. Signed-off-by: Andy Fleming Tested-by: Simon Glass --- v2: - Update to keep BUILD_NBUILDS builds in flight, rather than batching - Clean up style things - Defer error output until build completion to make output *slightly* more readable Performance tests (highly unscientific): ./MAKEALL 83xx (original) real 6m27.123s user 12m59.342s sys 2m2.528s BUILD_NBUILDS=2 ./MAKEALL 83xx real 4m54.854s user 12m10.336s sys 1m55.957s BUILD_NBUILDS=4 ./MAKEALL 83xx real 3m17.039s user 11m9.596s sys 1m40.327s BUILD_NBUILDS=8 ./MAKEALL 83xx real 2m44.366s user 10m38.658s sys 1m32.425s Amusingly, this was the best configuration I found so far: BUILD_NBUILDS=50 BUILD_NCPUS=1 ./MAKEALL 83xx real 2m1.252s user 10m49.617s sys 1m36.176s MAKEALL | 115 +++++++++++++++++++++++++++++++++++++++++++++++++++++++------- 1 files changed, 102 insertions(+), 13 deletions(-) diff --git a/MAKEALL b/MAKEALL index 95b7cd3..c14a1da 100755 --- a/MAKEALL +++ b/MAKEALL @@ -31,6 +31,7 @@ usage() CROSS_COMPILE cross-compiler toolchain prefix (default: "") MAKEALL_LOGDIR output all logs to here (default: ./LOG/) BUILD_DIR output build directory (default: ./) + BUILD_NBUILDS number of parallel targets (default: 1) Examples: - build all Power Architecture boards: @@ -160,11 +161,20 @@ else LOG_DIR="LOG" fi -if [ ! "${BUILD_DIR}" ] ; then - BUILD_DIR="." +: ${BUILD_NBUILDS:=1} + +if [ "${BUILD_NBUILDS}" -gt 1 ] ; then + : ${BUILD_DIR:=./build} + mkdir -p "${BUILD_DIR}/ERR" + find "${BUILD_DIR}/ERR/" -type f -exec rm -f {} + fi -[ -d ${LOG_DIR} ] || mkdir ${LOG_DIR} || exit 1 +: ${BUILD_DIR:=.} + +OUTPUT_PREFIX="${BUILD_DIR}" + +[ -d ${LOG_DIR} ] || mkdir "${LOG_DIR}" || exit 1 +find "${LOG_DIR}/" -type f -exec rm -f {} + LIST="" @@ -172,6 +182,8 @@ LIST="" ERR_CNT=0 ERR_LIST="" TOTAL_CNT=0 +CURRENT_COUNT=0 +OLDEST_IDX=1 RC=0 # Helper funcs for parsing boards.cfg @@ -485,31 +497,56 @@ LIST_nds32="$(boards_by_arch nds32)" #----------------------------------------------------------------------- +DONE_PREFIX="${LOG_DIR}/._build_" +SKIP_PREFIX="${LOG_DIR}/._skip_" + build_target() { target=$1 + build_idx=$2 + + if [ "$BUILD_NBUILDS" -gt 1 ] ; then + output_dir="${OUTPUT_PREFIX}/${target}" + mkdir -p "${output_dir}" + else + output_dir="${OUTPUT_PREFIX}" + fi + + export BUILD_DIR="${output_dir}" ${MAKE} distclean >/dev/null ${MAKE} -s ${target}_config - ${MAKE} ${JOBS} all 2>&1 >${LOG_DIR}/$target.MAKELOG \ - | tee ${LOG_DIR}/$target.ERR + ${MAKE} ${JOBS} all \ + >${LOG_DIR}/$target.MAKELOG 2> ${LOG_DIR}/$target.ERR # Check for 'make' errors if [ ${PIPESTATUS[0]} -ne 0 ] ; then RC=1 fi - if [ -s ${LOG_DIR}/$target.ERR ] ; then - ERR_CNT=$((ERR_CNT + 1)) - ERR_LIST="${ERR_LIST} $target" + if [ "$BUILD_NBUILDS" -gt 1 ] ; then + ${MAKE} tidy + + if [ -s ${LOG_DIR}/${target}.ERR ] ; then + touch ${OUTPUT_PREFIX}/ERR/${target} + else + rm ${LOG_DIR}/${target}.ERR + fi else - rm ${LOG_DIR}/$target.ERR + if [ -s ${LOG_DIR}/${target}.ERR ] ; then + : $(( ERR_CNT += 1 )) + ERR_LIST="${ERR_LIST} $target" + else + rm ${LOG_DIR}/${target}.ERR + fi fi - TOTAL_CNT=$((TOTAL_CNT + 1)) - - ${CROSS_COMPILE}size ${BUILD_DIR}/u-boot \ + ${CROSS_COMPILE}size ${output_dir}/u-boot \ | tee -a ${LOG_DIR}/$target.MAKELOG + + [ -e "${LOG_DIR}/${target}.ERR" ] && cat "${LOG_DIR}/${target}.ERR" + + touch "${DONE_PREFIX}${build_idx}" } build_targets() { for t in "$@" ; do @@ -523,7 +560,49 @@ build_targets() { if [ -n "${list}" ] ; then build_targets ${list} else - build_target ${t} + : $((TOTAL_CNT += 1)) + : $((CURRENT_COUNT += 1)) + rm -f "${DONE_PREFIX}${TOTAL_CNT}" + rm -f "${SKIP_PREFIX}${TOTAL_CNT}" + build_target ${t} ${TOTAL_CNT} & + fi + + # We maintain a running count of all the builds we have done. + # Each finished build will have a file called ${DONE_PREFIX}${n}, + # where n is the index of the build. Each build + # we've already noted as finished will have ${SKIP_PREFIX}${n}. + # We track the current index via TOTAL_CNT, and the oldest + # index. When we exceed the maximum number of parallel builds, + # We look from oldest to current for builds that have completed, + # and update the current count and oldest index as appropriate. + # If we've gone through the entire list, wait a second, and + # reprocess the entire list until we find a build that has + # completed + if [ ${CURRENT_COUNT} -ge ${BUILD_NBUILDS} ] ; then + search_idx=${OLDEST_IDX} + while true; do + if [ -e "${DONE_PREFIX}${search_idx}" ] ; then + : $(( CURRENT_COUNT-- )) + [ ${OLDEST_IDX} -eq ${search_idx} ] && + : $(( OLDEST_IDX++ )) + + # Only want to count it once + rm -f "${DONE_PREFIX}${search_idx}" + touch "${SKIP_PREFIX}${search_idx}" + elif [ -e "${SKIP_PREFIX}${search_idx}" ] ; then + [ ${OLDEST_IDX} -eq ${search_idx} ] && + : $(( OLDEST_IDX++ )) + fi + if [ ${search_idx} -ge ${TOTAL_CNT} ] ; then + if [ ${CURRENT_COUNT} -ge ${BUILD_NBUILDS} ] ; then + search_idx=${OLDEST_IDX} + sleep 1 + else + break + fi + fi + : $(( search_idx++ )) + done fi done } @@ -531,6 +610,15 @@ build_targets() { #----------------------------------------------------------------------- print_stats() { + for ((build_num=1; ${build_num} <= ${TOTAL_CNT} ; build_num++)) ; do + rm -f "${DONE_PREFIX}${build_num}" + rm -f "${SKIP_PREFIX}${build_num}" + done + + if [ "$BUILD_NBUILDS" -gt 1 ] ; then + ERR_LIST=$(ls ${OUTPUT_PREFIX}/ERR/) + ERR_CNT=`ls -1 ${OUTPUT_PREFIX}/ERR/ | wc | awk '{print $1}'` + fi echo "" echo "--------------------- SUMMARY ----------------------------" echo "Boards compiled: ${TOTAL_CNT}" @@ -549,3 +637,4 @@ set -- ${SELECTED} "$@" # run PowerPC by default [ $# = 0 ] && set -- powerpc build_targets "$@" +wait