| Message ID | 1400755421-9996-1-git-send-email-l.majewski@samsung.com |
|---|---|
| State | Superseded |
| Delegated to: | Marek Vasut |
| Headers | show |
Hi Lukasz, On 22 May 2014 00:43, Lukasz Majewski <l.majewski@samsung.com> wrote: > This commit adds test script for testing if any commit has introduced > regression to the DFU. > > It uses md5 to test if sent and received file is correct. > The test detailed description is available at DESCRIPTION.TXT file. > > Signed-off-by: Lukasz Majewski <l.majewski@samsung.com> > --- > test/dfu/DESCRIPTION.TXT | 27 ++++++++++++++ This sounds like an important and useful test, but I have a few comments (coming from someone with little knowledge of dfu) Would README be a better name? > test/dfu/dfu_gadget_test.sh | 86 +++++++++++++++++++++++++++++++++++++++++++ > 2 files changed, 113 insertions(+) > create mode 100644 test/dfu/DESCRIPTION.TXT > create mode 100755 test/dfu/dfu_gadget_test.sh > > diff --git a/test/dfu/DESCRIPTION.TXT b/test/dfu/DESCRIPTION.TXT > new file mode 100644 > index 0000000..48da06b > --- /dev/null > +++ b/test/dfu/DESCRIPTION.TXT > @@ -0,0 +1,27 @@ > +DFU TEST CASE DESCRIPTION: > + > +For running test script one needs to create: > +./log and ./bkp > + > +One also need to generate with dd following files: > + > +dat_* > + > +(e.g. dat_127B.img dat_128B.img dat_129B.img dat_33M.img dat_4095B.img > +dat_4096B.img dat_4097B.img dat_63B.img dat_64B.img dat_65B.img > +dat_960.img dat_97M.img) How do you do that? Is this intended for use with a particular board? I think you are missing some info here. > + > +The size is only important (so if=/dev/urandom), especially corner cases of > +DFU EP0's size of packet (64B). This is very helpful for fixing UDC driver > +regressions. > + > +Moreover on a target device the "dfu_alt_info" env variable should be extended > +to have "dfu_test.bin fat 0 6;" \ entry [1]. What is [1] ? > +One can use fat, ext4 or any other supported file system. > + > +As a prerequisite one must also create proper partition with file system on it. > + > +Example usage: > +./dfu_gadget_test.sh 11 > + > +where 11 is the mumber of alt setting corresponding to entry [1]. > \ No newline at end of file I'm really none-the-wiser as to how this works - I think you need to add a bit more detail. > diff --git a/test/dfu/dfu_gadget_test.sh b/test/dfu/dfu_gadget_test.sh > new file mode 100755 > index 0000000..ebde2ff > --- /dev/null > +++ b/test/dfu/dfu_gadget_test.sh > @@ -0,0 +1,86 @@ > +#! /bin/bash > +set -e # any command return not equal to zero > +clear > + > +DIR=./ > +SUFFIX=img > +RCV_DIR=rcv/ > +LOG_FILE=./log/log-`date +%d-%m-%Y_%H-%M-%S` > + > +cleanup () { > + rm -rf $RCV_DIR > +} > + > +die () { > + printf " \33[31m FAILED \33[0m \n" Perhaps define $RED or whatever at the top and use those here? > + cleanup > + exit 1 > +} > + > +calculate_md5sum () { > + MD5SUM=`md5sum $1` > + MD5SUM=`echo $MD5SUM | cut -d ' ' -f1` > + echo "md5sum:"$MD5SUM > +} > + > +dfu_test_file () { I suggest putting the argument in a variable here, instead of using $1 throughout the function. > + printf "\33[32m========================================================================================= \33[0m\n" > + printf "File:\33[32m %s \33[0m\n" $1 > + > + # dfu-util -D $1 -a $TARGET_ALT_SETTING > /dev/null 2>&1 Remove this line? > + dfu-util -D $1 -a $TARGET_ALT_SETTING >> $LOG_FILE 2>&1 || die $? > + > + echo -n "TX: " > + calculate_md5sum $1 > + > + MD5_TX=$MD5SUM > + > + N_FILE=$DIR$RCV_DIR${1:2}"_rcv" > + > + # dfu-util -U $N_FILE -a $TARGET_ALT_SETTING > /dev/null 2>&1 And again > + dfu-util -U $N_FILE -a $TARGET_ALT_SETTING >> $LOG_FILE 2>&1 || die $? > + > + echo -n "RX: " > + calculate_md5sum $N_FILE > + MD5_RX=$MD5SUM > + > + if [ "$MD5_TX" == "$MD5_RX" ] > + then > + printf " \33[32m -------> OK \33[0m \n" > + else outdent > + printf " \33[31m -------> FAILED \33[0m \n" > + cleanup > + exit 1 > + fi > + > +# echo $N_FILE > +} > + > +printf "\33[32m========================================================================================= \33[0m\n" > +echo "DFU EP0 transmission test program" > +echo "Trouble shoot -> disable DBG (even the KERN_DEBUG) in the UDC driver" > +echo "@ -> TRATS # dfu mmc 0" > +mkdir -p $RCV_DIR > +touch $LOG_FILE > + > +if [ $# -eq 0 ] > +then > + printf " \33[31m Please pass alt setting number!! \33[0m \n" > + exit 0 > +fi > + > +TARGET_ALT_SETTING=$1 > + > +if [ -n "$2" ] > +then > + dfu_test_file $2 > +else > + for file in $DIR*.$SUFFIX > + do > + dfu_test_file $file > + done > +fi > + > +cleanup > + > +exit 0 > \ No newline at end of file Why not? > -- > 1.7.10.4 > Regards, Simon
Hi Simon, > Hi Lukasz, > > On 22 May 2014 00:43, Lukasz Majewski <l.majewski@samsung.com> wrote: > > This commit adds test script for testing if any commit has > > introduced regression to the DFU. > > > > It uses md5 to test if sent and received file is correct. > > The test detailed description is available at DESCRIPTION.TXT file. > > > > Signed-off-by: Lukasz Majewski <l.majewski@samsung.com> > > --- > > test/dfu/DESCRIPTION.TXT | 27 ++++++++++++++ > > This sounds like an important and useful test, but I have a few > comments (coming from someone with little knowledge of dfu) > > Would README be a better name? I think it will be better. > > > test/dfu/dfu_gadget_test.sh | 86 > > +++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 113 > > insertions(+) create mode 100644 test/dfu/DESCRIPTION.TXT > > create mode 100755 test/dfu/dfu_gadget_test.sh > > > > diff --git a/test/dfu/DESCRIPTION.TXT b/test/dfu/DESCRIPTION.TXT > > new file mode 100644 > > index 0000000..48da06b > > --- /dev/null > > +++ b/test/dfu/DESCRIPTION.TXT > > @@ -0,0 +1,27 @@ > > +DFU TEST CASE DESCRIPTION: > > + > > +For running test script one needs to create: > > +./log and ./bkp > > + > > +One also need to generate with dd following files: > > + > > +dat_* > > + > > +(e.g. dat_127B.img dat_128B.img dat_129B.img dat_33M.img > > dat_4095B.img +dat_4096B.img dat_4097B.img dat_63B.img > > dat_64B.img dat_65B.img +dat_960.img dat_97M.img) > > How do you do that? I generate them with using dd if=/dev/urandom of=/dev/null bs=XX count=YY Generation of those test images can be easily scripted. > Is this intended for use with a particular board? No. Those values shall be applicable to all boards which use DFU. For example, since DFU uses the 64B as packet size, it is important to test +1/-1 size. With those tests one can also test UDC driver for regression. Other file sizes (like 33MiB or 97MiB) are picked randomly to check if transferring large or very large files work. > I think you are missing some info here. > > > + > > +The size is only important (so if=/dev/urandom), especially corner > > cases of +DFU EP0's size of packet (64B). This is very helpful for > > fixing UDC driver +regressions. > > + > > +Moreover on a target device the "dfu_alt_info" env variable should > > be extended +to have "dfu_test.bin fat 0 6;" \ entry [1]. > > What is [1] ? [1] is the entry: "dfu_test.bin fat 0 6;", which should be added to the "dfu_alt_info" environment variable to define one special alt setting for testing purposes. > > > +One can use fat, ext4 or any other supported file system. > > + > > +As a prerequisite one must also create proper partition with file > > system on it. + > > +Example usage: > > +./dfu_gadget_test.sh 11 > > + > > +where 11 is the mumber of alt setting corresponding to entry [1]. > > \ No newline at end of file > > I'm really none-the-wiser as to how this works - I think you need to > add a bit more detail. Ok. In short - the parameter which the script ./dfu_gadget_test.sh accepts is 11 (therefore we must type ./dfu_gadget_test.sh 11 > > > diff --git a/test/dfu/dfu_gadget_test.sh > > b/test/dfu/dfu_gadget_test.sh new file mode 100755 > > index 0000000..ebde2ff > > --- /dev/null > > +++ b/test/dfu/dfu_gadget_test.sh > > @@ -0,0 +1,86 @@ > > +#! /bin/bash > > +set -e # any command return not equal to zero > > +clear > > + > > +DIR=./ > > +SUFFIX=img > > +RCV_DIR=rcv/ > > +LOG_FILE=./log/log-`date +%d-%m-%Y_%H-%M-%S` > > + > > +cleanup () { > > + rm -rf $RCV_DIR > > +} > > + > > +die () { > > + printf " \33[31m FAILED \33[0m \n" > > Perhaps define $RED or whatever at the top and use those here? Good point, thanks. > > > + cleanup > > + exit 1 > > +} > > + > > +calculate_md5sum () { > > + MD5SUM=`md5sum $1` > > + MD5SUM=`echo $MD5SUM | cut -d ' ' -f1` > > + echo "md5sum:"$MD5SUM > > +} > > + > > +dfu_test_file () { > > I suggest putting the argument in a variable here, instead of using $1 > throughout the function. OK. > > > + printf > > "\33[32m========================================================================================= > > \33[0m\n" > > + printf "File:\33[32m %s \33[0m\n" $1 > > + > > + # dfu-util -D $1 -a $TARGET_ALT_SETTING > /dev/null 2>&1 > > Remove this line? Yes, agree. This line seems like a remain after testing. > > > + dfu-util -D $1 -a $TARGET_ALT_SETTING >> $LOG_FILE 2>&1 || die > > $? + > > + echo -n "TX: " > > + calculate_md5sum $1 > > + > > + MD5_TX=$MD5SUM > > + > > + N_FILE=$DIR$RCV_DIR${1:2}"_rcv" > > + > > + # dfu-util -U $N_FILE -a $TARGET_ALT_SETTING > /dev/null 2>&1 > > And again OK. > > > + dfu-util -U $N_FILE -a $TARGET_ALT_SETTING >> $LOG_FILE 2>&1 > > || die $? + > > + echo -n "RX: " > > + calculate_md5sum $N_FILE > > + MD5_RX=$MD5SUM > > + > > + if [ "$MD5_TX" == "$MD5_RX" ] > > + then > > + printf " \33[32m -------> OK \33[0m \n" > > + else > > outdent > > > + printf " \33[31m -------> FAILED \33[0m \n" > > + cleanup > > + exit 1 > > + fi > > + > > +# echo $N_FILE > > +} > > + > > +printf > > "\33[32m========================================================================================= > > \33[0m\n" +echo "DFU EP0 transmission test program" +echo "Trouble > > shoot -> disable DBG (even the KERN_DEBUG) in the UDC driver" +echo > > "@ -> TRATS # dfu mmc 0" +mkdir -p $RCV_DIR > > +touch $LOG_FILE > > + > > +if [ $# -eq 0 ] > > +then > > + printf " \33[31m Please pass alt setting number!! \33[0m > > \n" > > + exit 0 > > +fi > > + > > +TARGET_ALT_SETTING=$1 > > + > > +if [ -n "$2" ] > > +then > > + dfu_test_file $2 > > +else > > + for file in $DIR*.$SUFFIX > > + do > > + dfu_test_file $file > > + done > > +fi > > + > > +cleanup > > + > > +exit 0 > > \ No newline at end of file > > Why not? OK. > > > -- > > 1.7.10.4 > > > > Regards, > Simon
On 05/23/2014 06:51 PM, Simon Glass wrote: > Hi Lukasz, > > On 22 May 2014 00:43, Lukasz Majewski <l.majewski@samsung.com> wrote: >> This commit adds test script for testing if any commit has introduced >> regression to the DFU. >> >> It uses md5 to test if sent and received file is correct. >> The test detailed description is available at DESCRIPTION.TXT file. >> >> Signed-off-by: Lukasz Majewski <l.majewski@samsung.com> >> --- >> test/dfu/DESCRIPTION.TXT | 27 ++++++++++++++ >> diff --git a/test/dfu/DESCRIPTION.TXT b/test/dfu/DESCRIPTION.TXT >> new file mode 100644 >> index 0000000..48da06b >> --- /dev/null >> +++ b/test/dfu/DESCRIPTION.TXT >> @@ -0,0 +1,27 @@ >> +DFU TEST CASE DESCRIPTION: >> + >> +For running test script one needs to create: >> +./log and ./bkp >> + >> +One also need to generate with dd following files: >> + >> +dat_* >> + >> +(e.g. dat_127B.img dat_128B.img dat_129B.img dat_33M.img dat_4095B.img >> +dat_4096B.img dat_4097B.img dat_63B.img dat_64B.img dat_65B.img >> +dat_960.img dat_97M.img) > > How do you do that? Is this intended for use with a particular board? > I think you are missing some info here. I think the test script should generate its own test files, so the user can simply run it without having to manually set up a bunch of stuff first. One important test-case is files that are larger than CONFIG_SYS_DFU_DATA_BUF_SIZE. Is dat_97M.img above intended to be 97MB? If so, that's probably covered. Testing large sizes that both are and are-not multiples of CONFIG_SYS_DFU_DATA_BUF_SIZE would be useful.
Hi, On 29 May 2014 09:58, Stephen Warren <swarren@wwwdotorg.org> wrote: > On 05/23/2014 06:51 PM, Simon Glass wrote: >> Hi Lukasz, >> >> On 22 May 2014 00:43, Lukasz Majewski <l.majewski@samsung.com> wrote: >>> This commit adds test script for testing if any commit has introduced >>> regression to the DFU. >>> >>> It uses md5 to test if sent and received file is correct. >>> The test detailed description is available at DESCRIPTION.TXT file. >>> >>> Signed-off-by: Lukasz Majewski <l.majewski@samsung.com> >>> --- >>> test/dfu/DESCRIPTION.TXT | 27 ++++++++++++++ > >>> diff --git a/test/dfu/DESCRIPTION.TXT b/test/dfu/DESCRIPTION.TXT >>> new file mode 100644 >>> index 0000000..48da06b >>> --- /dev/null >>> +++ b/test/dfu/DESCRIPTION.TXT >>> @@ -0,0 +1,27 @@ >>> +DFU TEST CASE DESCRIPTION: >>> + >>> +For running test script one needs to create: >>> +./log and ./bkp >>> + >>> +One also need to generate with dd following files: >>> + >>> +dat_* >>> + >>> +(e.g. dat_127B.img dat_128B.img dat_129B.img dat_33M.img dat_4095B.img >>> +dat_4096B.img dat_4097B.img dat_63B.img dat_64B.img dat_65B.img >>> +dat_960.img dat_97M.img) >> >> How do you do that? Is this intended for use with a particular board? >> I think you are missing some info here. > > I think the test script should generate its own test files, so the user > can simply run it without having to manually set up a bunch of stuff first. Agreed. As a general rule it would be good if the tests would run through all possible options and not require user involvement. I'm not sure if that is possible in this case though. > > One important test-case is files that are larger than > CONFIG_SYS_DFU_DATA_BUF_SIZE. Is dat_97M.img above intended to be 97MB? > If so, that's probably covered. Testing large sizes that both are and > are-not multiples of CONFIG_SYS_DFU_DATA_BUF_SIZE would be useful. Regards, Simon
Hi Simon, > Hi, > > On 29 May 2014 09:58, Stephen Warren <swarren@wwwdotorg.org> wrote: > > On 05/23/2014 06:51 PM, Simon Glass wrote: > >> Hi Lukasz, > >> > >> On 22 May 2014 00:43, Lukasz Majewski <l.majewski@samsung.com> > >> wrote: > >>> This commit adds test script for testing if any commit has > >>> introduced regression to the DFU. > >>> > >>> It uses md5 to test if sent and received file is correct. > >>> The test detailed description is available at DESCRIPTION.TXT > >>> file. > >>> > >>> Signed-off-by: Lukasz Majewski <l.majewski@samsung.com> > >>> --- > >>> test/dfu/DESCRIPTION.TXT | 27 ++++++++++++++ > > > >>> diff --git a/test/dfu/DESCRIPTION.TXT b/test/dfu/DESCRIPTION.TXT > >>> new file mode 100644 > >>> index 0000000..48da06b > >>> --- /dev/null > >>> +++ b/test/dfu/DESCRIPTION.TXT > >>> @@ -0,0 +1,27 @@ > >>> +DFU TEST CASE DESCRIPTION: > >>> + > >>> +For running test script one needs to create: > >>> +./log and ./bkp > >>> + > >>> +One also need to generate with dd following files: > >>> + > >>> +dat_* > >>> + > >>> +(e.g. dat_127B.img dat_128B.img dat_129B.img dat_33M.img > >>> dat_4095B.img +dat_4096B.img dat_4097B.img dat_63B.img > >>> dat_64B.img dat_65B.img +dat_960.img dat_97M.img) > >> > >> How do you do that? Is this intended for use with a particular > >> board? I think you are missing some info here. > > > > I think the test script should generate its own test files, so the > > user can simply run it without having to manually set up a bunch of > > stuff first. > > Agreed. As a general rule it would be good if the tests would run > through all possible options and not require user involvement. I'm not > sure if that is possible in this case though. I could provide a special script to automatize the work. Either in the script itself or via configuration file it could be possible to specify files' size to be used. Running this script would assure that each user would have the same environment. > > > > > One important test-case is files that are larger than > > CONFIG_SYS_DFU_DATA_BUF_SIZE. Is dat_97M.img above intended to be > > 97MB? If so, that's probably covered. Testing large sizes that both > > are and are-not multiples of CONFIG_SYS_DFU_DATA_BUF_SIZE would be > > useful. Yes, this is the case. > > Regards, > Simon
diff --git a/test/dfu/DESCRIPTION.TXT b/test/dfu/DESCRIPTION.TXT new file mode 100644 index 0000000..48da06b --- /dev/null +++ b/test/dfu/DESCRIPTION.TXT @@ -0,0 +1,27 @@ +DFU TEST CASE DESCRIPTION: + +For running test script one needs to create: +./log and ./bkp + +One also need to generate with dd following files: + +dat_* + +(e.g. dat_127B.img dat_128B.img dat_129B.img dat_33M.img dat_4095B.img +dat_4096B.img dat_4097B.img dat_63B.img dat_64B.img dat_65B.img +dat_960.img dat_97M.img) + +The size is only important (so if=/dev/urandom), especially corner cases of +DFU EP0's size of packet (64B). This is very helpful for fixing UDC driver +regressions. + +Moreover on a target device the "dfu_alt_info" env variable should be extended +to have "dfu_test.bin fat 0 6;" \ entry [1]. +One can use fat, ext4 or any other supported file system. + +As a prerequisite one must also create proper partition with file system on it. + +Example usage: +./dfu_gadget_test.sh 11 + +where 11 is the mumber of alt setting corresponding to entry [1]. \ No newline at end of file diff --git a/test/dfu/dfu_gadget_test.sh b/test/dfu/dfu_gadget_test.sh new file mode 100755 index 0000000..ebde2ff --- /dev/null +++ b/test/dfu/dfu_gadget_test.sh @@ -0,0 +1,86 @@ +#! /bin/bash +set -e # any command return not equal to zero +clear + +DIR=./ +SUFFIX=img +RCV_DIR=rcv/ +LOG_FILE=./log/log-`date +%d-%m-%Y_%H-%M-%S` + +cleanup () { + rm -rf $RCV_DIR +} + +die () { + printf " \33[31m FAILED \33[0m \n" + cleanup + exit 1 +} + +calculate_md5sum () { + MD5SUM=`md5sum $1` + MD5SUM=`echo $MD5SUM | cut -d ' ' -f1` + echo "md5sum:"$MD5SUM +} + +dfu_test_file () { + printf "\33[32m========================================================================================= \33[0m\n" + printf "File:\33[32m %s \33[0m\n" $1 + + # dfu-util -D $1 -a $TARGET_ALT_SETTING > /dev/null 2>&1 + dfu-util -D $1 -a $TARGET_ALT_SETTING >> $LOG_FILE 2>&1 || die $? + + echo -n "TX: " + calculate_md5sum $1 + + MD5_TX=$MD5SUM + + N_FILE=$DIR$RCV_DIR${1:2}"_rcv" + + # dfu-util -U $N_FILE -a $TARGET_ALT_SETTING > /dev/null 2>&1 + dfu-util -U $N_FILE -a $TARGET_ALT_SETTING >> $LOG_FILE 2>&1 || die $? + + echo -n "RX: " + calculate_md5sum $N_FILE + MD5_RX=$MD5SUM + + if [ "$MD5_TX" == "$MD5_RX" ] + then + printf " \33[32m -------> OK \33[0m \n" + else + printf " \33[31m -------> FAILED \33[0m \n" + cleanup + exit 1 + fi + +# echo $N_FILE +} + +printf "\33[32m========================================================================================= \33[0m\n" +echo "DFU EP0 transmission test program" +echo "Trouble shoot -> disable DBG (even the KERN_DEBUG) in the UDC driver" +echo "@ -> TRATS # dfu mmc 0" +mkdir -p $RCV_DIR +touch $LOG_FILE + +if [ $# -eq 0 ] +then + printf " \33[31m Please pass alt setting number!! \33[0m \n" + exit 0 +fi + +TARGET_ALT_SETTING=$1 + +if [ -n "$2" ] +then + dfu_test_file $2 +else + for file in $DIR*.$SUFFIX + do + dfu_test_file $file + done +fi + +cleanup + +exit 0
This commit adds test script for testing if any commit has introduced regression to the DFU. It uses md5 to test if sent and received file is correct. The test detailed description is available at DESCRIPTION.TXT file. Signed-off-by: Lukasz Majewski <l.majewski@samsung.com> --- test/dfu/DESCRIPTION.TXT | 27 ++++++++++++++ test/dfu/dfu_gadget_test.sh | 86 +++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 113 insertions(+) create mode 100644 test/dfu/DESCRIPTION.TXT create mode 100755 test/dfu/dfu_gadget_test.sh \ No newline at end of file