[v2] boot_test.sh: Add SMC BMC support

Your BMC needs a special debug image flashed to use this, the exact
image and methods aren't something I can publish here, but if you work
for IBM or SMC you can find out from the right sources.

A few things are needed to move around to be able to flash to a SMC BMC.

For a start, the SSH daemon will only accept connections after a special
incantation (which I also can't share), but you should put that in the
~/.skiboot_boot_tests file along with some other default login information
we don't publicise too broadly (because Security Through Obscurity is
*obviously* a good idea....)

We also can't just directly "ssh /bin/true", we need an expect script,
and we can't scp, but we can anonymous rsync!

You also need a pflash binary to copy over.

Signed-off-by: Stewart Smith <stewart@linux.vnet.ibm.com>
---
Changes in v2:
- up the timeout in expect script, allows flashing of full pnor
- use mboxctl
---
 external/boot-tests/bmc_support.sh |  10 +++
 external/boot-tests/boot_test.sh   |   7 --
 external/boot-tests/fsp_support.sh |  11 ++++
 external/boot-tests/smc_support.sh | 128 +++++++++++++++++++++++++++++++++++++
 4 files changed, 149 insertions(+), 7 deletions(-)
 create mode 100644 external/boot-tests/smc_support.sh

On Mon, 2017-06-05 at 15:02 +1000, Stewart Smith wrote:
> Your BMC needs a special debug image flashed to use this, the exact
> image and methods aren't something I can publish here, but if you work
> for IBM or SMC you can find out from the right sources.
> 
> A few things are needed to move around to be able to flash to a SMC BMC.
> 
> For a start, the SSH daemon will only accept connections after a special
> incantation (which I also can't share), but you should put that in the
> ~/.skiboot_boot_tests file along with some other default login information
> we don't publicise too broadly (because Security Through Obscurity is
> *obviously* a good idea....)
> 
> We also can't just directly "ssh /bin/true", we need an expect script,
> and we can't scp, but we can anonymous rsync!
> 
> You also need a pflash binary to copy over.
> 
> Signed-off-by: Stewart Smith <stewart@linux.vnet.ibm.com>

Reviewed-by: Cyril Bur <cyril.bur@au1.ibm.com>

> ---
> Changes in v2:
> - up the timeout in expect script, allows flashing of full pnor
> - use mboxctl
> ---
>  external/boot-tests/bmc_support.sh |  10 +++
>  external/boot-tests/boot_test.sh   |   7 --
>  external/boot-tests/fsp_support.sh |  11 ++++
>  external/boot-tests/smc_support.sh | 128 +++++++++++++++++++++++++++++++++++++
>  4 files changed, 149 insertions(+), 7 deletions(-)
>  create mode 100644 external/boot-tests/smc_support.sh
> 
> diff --git a/external/boot-tests/bmc_support.sh b/external/boot-tests/bmc_support.sh
> index a61df25..82b9e8e 100644
> --- a/external/boot-tests/bmc_support.sh
> +++ b/external/boot-tests/bmc_support.sh
> @@ -19,6 +19,10 @@ STRIP_CONTROL=0
>  # How do we SSH/SCP in?
>  SSHCMD="sshpass -e ssh -l $SSHUSER -o LogLevel=quiet -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no $target";
>  
> +function sshcmd {
> +	$SSHCMD $*;
> +}
> +
>  # remotecp file target target_location
>  function remotecp {
>  	sshpass -e ssh -o User=$SSHUSER -o LogLevel=quiet -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no $2 dd of=$3 < $1;
> @@ -132,6 +136,12 @@ function boot_firmware {
>  }
>  
>  function machine_sanity_test {
> +    sshcmd true;
> +    if [ $? -ne 0 ]; then
> +	echo "$target: Failed to SSH to $target..."
> +        echo "$target: Command was: $SSHCMD true"
> +	error "Try connecting manually to diagnose the issue."
> +    fi
>      # No further sanity tests for BMC machines.
>      true
>  }
> diff --git a/external/boot-tests/boot_test.sh b/external/boot-tests/boot_test.sh
> index 4b73914..5b3fc15 100755
> --- a/external/boot-tests/boot_test.sh
> +++ b/external/boot-tests/boot_test.sh
> @@ -104,13 +104,6 @@ function boot_test {
>  }
>  
>  function sanity_test {
> -    $SSHCMD true;
> -    if [ $? -ne 0 ]; then
> -	echo "$target: Failed to SSH to $target..."
> -        echo "$target: Command was: $SSHCMD true"
> -	error "Try connecting manually to diagnose the issue."
> -    fi
> -
>      $IPMI_COMMAND chassis power status > /dev/null;
>      if [ $? -ne 0 ]; then
>  	echo "$target: Failed to connect to $target with IPMI..."
> diff --git a/external/boot-tests/fsp_support.sh b/external/boot-tests/fsp_support.sh
> index c27f307..fc0d66d 100644
> --- a/external/boot-tests/fsp_support.sh
> +++ b/external/boot-tests/fsp_support.sh
> @@ -24,6 +24,10 @@ REMOTECPCMD="sshpass -e scp -o User=$SSHUSER -o LogLevel=quiet -o UserKnownHosts
>  
>  GET_PROFILE='. /etc/profile; test -e /home/dev/.profile && . /home/dev/.profile';
>  
> +function sshcmd {
> +	$SSHCMD $*;
> +}
> +
>  function is_off {
>      state=$($SSHCMD "$GET_PROFILE; smgr mfgState");
>      return $([ "$state" = "standby" ]);
> @@ -153,6 +157,13 @@ function boot_firmware {
>  }
>  
>  function machine_sanity_test {
> +    sshcmd true;
> +    if [ $? -ne 0 ]; then
> +	echo "$target: Failed to SSH to $target..."
> +        echo "$target: Command was: $SSHCMD true"
> +	error "Try connecting manually to diagnose the issue."
> +    fi
> +
>      $SSHCMD "$GET_PROFILE; test -d /nfs/bin"
>      if [ $? -ne 0 ]; then
>  	echo "$target: Failed to read /nfs/bin"
> diff --git a/external/boot-tests/smc_support.sh b/external/boot-tests/smc_support.sh
> new file mode 100644
> index 0000000..5e2a834
> --- /dev/null
> +++ b/external/boot-tests/smc_support.sh
> @@ -0,0 +1,128 @@
> +#Number of times to sleep
> +BOOT_TIMEOUT="5";
> +
> +#Username/password for ssh to BMC machines
> +SSHUSER=${SSHUSER:-ADMIN};
> +export SSHPASS=${SSHPASS:-ADMIN};
> +
> +#Username/password for IPMI
> +IPMI_AUTH="-U ${IPMI_USER:-ADMIN} -P ${IPMI_PASS:-ADMIN}"
> +PFLASH_TO_COPY=${PFLASH_TO_COPY:-}
> +PFLASH_BINARY=/tmp/pflash
> +
> +# Strip control characters from IPMI before grepping?
> +STRIP_CONTROL=0
> +
> +# How do we SSH/SCP in?
> +SSHCMD="sshpass -e ssh -l $SSHUSER -o LogLevel=quiet -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no $target";
> +
> +function sshcmd {
> +    # because BMC:
> +    $IPMI_COMMAND $SMC_PRESSHIPMICMD;
> +    expect -c "spawn $SSHCMD" -c "set timeout 600" -c "expect \"#\" { send \"$*\\r\" }" -c 'expect "#" { send "exit\r" }' -c 'wait';
> +}
> +
> +# remotecp file target target_location
> +function remotecp {
> +    rsync -av $1 rsync://$2/files/$3
> +}
> +
> +function is_off {
> +    return $([ "$($IPMI_COMMAND chassis power status)" = "Chassis Power is off" ]);
> +}
> +
> +function poweroff {
> +    $IPMI_COMMAND chassis power off
> +    # give it some time
> +    sleep 10
> +}
> +
> +function force_primary_side {
> +    true
> +}
> +
> +function flash {
> +	if [ ! -z "$PFLASH_TO_COPY" ]; then
> +		remotecp $PFLASH_TO_COPY $target pflash
> +		sshcmd chmod +x /tmp/rsync_file/pflash
> +		PFLASH_BINARY=/tmp/rsync_file/pflash
> +	fi
> +	if [ ! -z "$PNOR" ]; then
> +		remotecp $PNOR $target image.pnor;
> +	fi
> +        if [ "${LID[0]}" != "" ]; then
> +		remotecp ${LID[0]} $target skiboot.lid;
> +	fi
> +	if [ "${LID[1]}" != "" ]; then
> +		remotecp ${LID[1]} $target bootkernel
> +	fi
> +	if [ "${arbitrary_lid[1]}" != "" ]; then
> +		remotecp ${arbitrary_lid[1]} $target $(basename ${arbitrary_lid[1]})
> +	fi
> +
> +	if [ "$?" -ne "0" ] ; then
> +		error "Couldn't copy firmware image";
> +	fi
> +

I noticed that the logic is the same in bmc_support.sh but I'll mention
it here anyway - that $? test will only trip if the last transfer
fails. Since it needs fixing in bmc_support.sh anyway... *shrugs*

> +	# flash it
> +	if [ ! -z "$PNOR" ]; then
> +		msg "Flashing full PNOR"
> +		sshcmd "$PFLASH_BINARY -E -f -p /tmp/rsync_file/image.pnor"
> +		if [ "$?" -ne "0" ] ; then
> +			error "An unexpected pflash error has occurred";
> +		fi
> +		msg "Removing /tmp/rsync_file/image.pnor"
> +		sshcmd "rm /tmp/rsync_file/image.pnor"
> +	fi
> +
> +	if [ ! -z "${LID[0]}" ] ; then
> +		msg "Flashing PAYLOAD PNOR partition"
> +		sshcmd "$PFLASH_BINARY -e -f -P PAYLOAD -p /tmp/rsync_file/skiboot.lid"
> +		if [ "$?" -ne "0" ] ; then
> +                        error "An unexpected pflash error has occurred";
> +		fi
> +		msg "Removing /tmp/rsync_file/skiboot.lid"
> +		sshcmd "rm /tmp/skiboot.lid"
> +	fi
> +
> +        if [ ! -z "${LID[1]}" ] ; then
> +                msg "Flashing BOOTKERNEL PNOR partition"
> +                sshcmd "$PFLASH_BINARY -e -f -P BOOTKERNEL -p /tmp/rsync_file/bootkernel"
> +                if [ "$?" -ne "0" ] ; then
> +                        error "An unexpected pflash error has occurred";
> +		fi
> +		msg "Removing /tmp/rsync_file/bootkernel"
> +		sshcmd "rm /tmp/rsync_file/bootkernel"
> +        fi
> +
> +	if [ ! -z "${arbitrary_lid[0]}" -a ! -z "${arbitrary_lid[1]}" ] ; then
> +		msg "Flashing ${arbitrary_lid[0]} PNOR partition"
> +		sshcmd "$PFLASH_BINARY -e -f -P ${arbitrary_lid[0]} -p /tmp/rsync_file/$(basename ${arbitrary_lid[1]})"
> +                if [ "$?" -ne "0" ] ; then
> +                        error "An unexpected pflash error has occurred";
> +		fi
> +		msg "Removing /tmp/rsync_file/$(basename ${arbitrary_lid[1]})"
> +		sshcmd "rm /tmp/rsync_file/$(basename ${arbitrary_lid[1]})"
> +	fi
> +
> +	msg "Clearing mboxd caches..."
> +	sshcmd "/bin/mboxctl --clear-cache"
> +}
> +
> +function boot_firmware {
> +    	$IPMI_COMMAND chassis power on > /dev/null;
> +	i=0;
> +	while [ "$($IPMI_COMMAND chassis power status)" = "Chassis Power is off" -a \( "$i" -lt "$BOOT_TIMEOUT" \) ] ; do
> +		msg -n ".";
> +		sleep $BOOT_SLEEP_PERIOD;
> +		i=$(expr $i + 1);
> +	done
> +	if [ "$i" -eq "$BOOT_TIMEOUT" ] ; then
> +		error "Couldn't power on $target";
> +	fi
> +}
> +
> +function machine_sanity_test {
> +    # No further sanity tests for BMC machines.
> +    true
> +}

Stewart Smith <stewart@linux.vnet.ibm.com> writes:
> Your BMC needs a special debug image flashed to use this, the exact
> image and methods aren't something I can publish here, but if you work
> for IBM or SMC you can find out from the right sources.
>
> A few things are needed to move around to be able to flash to a SMC BMC.
>
> For a start, the SSH daemon will only accept connections after a special
> incantation (which I also can't share), but you should put that in the
> ~/.skiboot_boot_tests file along with some other default login information
> we don't publicise too broadly (because Security Through Obscurity is
> *obviously* a good idea....)
>
> We also can't just directly "ssh /bin/true", we need an expect script,
> and we can't scp, but we can anonymous rsync!
>
> You also need a pflash binary to copy over.
>
> Signed-off-by: Stewart Smith <stewart@linux.vnet.ibm.com>
> ---
> Changes in v2:
> - up the timeout in expect script, allows flashing of full pnor
> - use mboxctl

Merged to master as of 0efe38cc7a322d668e5b494cb4b464522950cec2