Patchwork [v4,2/2] qemu-ga: sample fsfreeze hooks

login
register
mail settings
Submitter Tomoki Sekiyama
Date Nov. 22, 2012, 2:15 a.m.
Message ID <20121122021549.9852.73727.stgit@melchior2.sdl.hitachi.co.jp>
Download mbox | patch
Permalink /patch/200925/
State New
Headers show

Comments

Tomoki Sekiyama - Nov. 22, 2012, 2:15 a.m.
Adds sample hook scripts for --fsfreeze-hook option of qemu-ga.
  - fsfreeze-hook : execute scripts in fsfreeze-hook.d/
  - fsfreeze-hook.d.sample/mysql-flush.sh : quiesce MySQL before snapshot

Signed-off-by: Tomoki Sekiyama <tomoki.sekiyama.qu@hitachi.com>
---
 docs/qemu-guest-agent/fsfreeze-hook                |   31 +++++++++++++
 .../fsfreeze-hook.d.sample/mysql-flush.sh          |   47 ++++++++++++++++++++
 2 files changed, 78 insertions(+)
 create mode 100755 docs/qemu-guest-agent/fsfreeze-hook
 create mode 100755 docs/qemu-guest-agent/fsfreeze-hook.d.sample/mysql-flush.sh
Luiz Capitulino - Nov. 22, 2012, 4:03 p.m.
On Thu, 22 Nov 2012 11:15:49 +0900
Tomoki Sekiyama <tomoki.sekiyama.qu@hitachi.com> wrote:

> Adds sample hook scripts for --fsfreeze-hook option of qemu-ga.
>   - fsfreeze-hook : execute scripts in fsfreeze-hook.d/
>   - fsfreeze-hook.d.sample/mysql-flush.sh : quiesce MySQL before snapshot
> 
> Signed-off-by: Tomoki Sekiyama <tomoki.sekiyama.qu@hitachi.com>
> ---
>  docs/qemu-guest-agent/fsfreeze-hook                |   31 +++++++++++++
>  .../fsfreeze-hook.d.sample/mysql-flush.sh          |   47 ++++++++++++++++++++
>  2 files changed, 78 insertions(+)
>  create mode 100755 docs/qemu-guest-agent/fsfreeze-hook
>  create mode 100755 docs/qemu-guest-agent/fsfreeze-hook.d.sample/mysql-flush.sh
> 
> diff --git a/docs/qemu-guest-agent/fsfreeze-hook b/docs/qemu-guest-agent/fsfreeze-hook
> new file mode 100755
> index 0000000..319f68c
> --- /dev/null
> +++ b/docs/qemu-guest-agent/fsfreeze-hook
> @@ -0,0 +1,31 @@
> +#!/bin/sh
> +
> +# This script is executed when a guest agent receives fsfreeze-freeze and
> +# fsfreeze-thaw command, if it is specified in --fsfreeze-hook (-F)
> +# option of qemu-ga or placed in default path (/etc/qemu/fsfreeze-hook).
> +# When the agent receives fsfreeze-freeze request, this script is issued with
> +# "freeze" argument before the filesystem is freezed. And for fsfreeze-thaw
> +# request, it is issued with "thaw" argument after filesystem is thawed.
> +
> +LOGFILE=/var/log/qga-fsfreeze-hook.log
> +FSFREEZE_D=$(dirname -- "$0")/fsfreeze-hook.d
> +
> +# Check whether file $1 is a backup or rpm-generated file and should be ignored
> +is_ignored_file() {
> +    case "$1" in
> +        *~ | *.bak | *.orig | *.rpmnew | *.rpmorig | *.rpmsave)
> +            return 0 ;;
> +    esac
> +    return 1
> +}
> +
> +# Iterate executables in directory "fsfreeze-hook.d" with the specified args
> +[ ! -d "$FSFREEZE_D" ] && exit 1
> +for file in "$FSFREEZE_D"/* ; do
> +    is_ignored_file "$file" && continue
> +    [ -x "$file" ] || continue
> +    echo "$(date): execute $file $@" >>$LOGFILE
> +    "$file" "$@" >>$LOGFILE 2>&1
> +    STATUS=$?
> +    echo "$(date): $file finished with status=$STATUS" >>$LOGFILE
> +done

execute_fsfreeze_hook() will fail the freeze process if this script fails. Two
comments:

 1. Do we want to fail the freeze process if one of the sub-scripts fails?
    If yes, then we have to exit 1 in the first failure

 2. The exit status of the script will echo's exit status. I doubt we want that

> diff --git a/docs/qemu-guest-agent/fsfreeze-hook.d.sample/mysql-flush.sh b/docs/qemu-guest-agent/fsfreeze-hook.d.sample/mysql-flush.sh
> new file mode 100755
> index 0000000..e6d7998
> --- /dev/null
> +++ b/docs/qemu-guest-agent/fsfreeze-hook.d.sample/mysql-flush.sh
> @@ -0,0 +1,47 @@
> +#!/bin/sh
> +
> +# Flush MySQL tables to the disk before the filesystem is freezed.
> +# At the same time, this keeps a read lock while the filesystem is freezed
> +# in order to avoid write accesses by the other clients.
> +
> +MYSQL="mysql -uroot" #"-prootpassword"
> +FIFO=/tmp/mysql-flush.fifo
> +
> +flush_and_wait() {
> +    printf "FLUSH TABLES WITH READ LOCK \\G\n"
> +    read < $FIFO
> +    printf "UNLOCK TABLES \\G\n"
> +}
> +
> +case "$1" in
> +    freeze)
> +        mkfifo $FIFO || exit 1
> +        flush_and_wait | $MYSQL &

Honest question: what happens if I don't have mysql installed?

> +        # wait until every block is flushed
> +        while [ "$(echo 'SHOW STATUS LIKE "Key_blocks_not_flushed"' |\
> +                 $MYSQL | tail -1 | cut -f 2)" -gt 0 ]; do
> +            sleep 1
> +        done
> +        # for InnoDB, wait until every log is flushed
> +        INNODB_STATUS=$(mktemp /tmp/mysql-flush.XXXXXX)
> +        [ $? -ne 0 ] && exit 2
> +        trap "rm -f $INNODB_STATUS" SIGINT
> +        while :; do
> +            printf "SHOW ENGINE INNODB STATUS \\G" | $MYSQL > $INNODB_STATUS
> +            LOG_CURRENT=$(grep 'Log sequence number' $INNODB_STATUS |\
> +                          tr -s ' ' | cut -d' ' -f4)
> +            LOG_FLUSHED=$(grep 'Log flushed up to' $INNODB_STATUS |\
> +                          tr -s ' ' | cut -d' ' -f5)
> +            [ "$LOG_CURRENT" = "$LOG_FLUSHED" ] && break
> +            sleep 1
> +        done
> +        rm -f $INNODB_STATUS
> +        ;;
> +
> +    thaw)
> +        [ ! -p $FIFO ] && exit 1
> +        echo > $FIFO
> +        rm -f $FIFO
> +        ;;
> +esac
> +
> 
>
Tomoki Sekiyama - Nov. 26, 2012, 11:49 a.m.
On 2012/11/23 1:03, Luiz Capitulino wrote:
> On Thu, 22 Nov 2012 11:15:49 +0900
> Tomoki Sekiyama <tomoki.sekiyama.qu@hitachi.com> wrote:
> 
>> Adds sample hook scripts for --fsfreeze-hook option of qemu-ga.
>>   - fsfreeze-hook : execute scripts in fsfreeze-hook.d/
>>   - fsfreeze-hook.d.sample/mysql-flush.sh : quiesce MySQL before snapshot
<snip>
>> +# Iterate executables in directory "fsfreeze-hook.d" with the specified args
>> +[ ! -d "$FSFREEZE_D" ] && exit 1
>> +for file in "$FSFREEZE_D"/* ; do
>> +    is_ignored_file "$file" && continue
>> +    [ -x "$file" ] || continue
>> +    echo "$(date): execute $file $@" >>$LOGFILE
>> +    "$file" "$@" >>$LOGFILE 2>&1
>> +    STATUS=$?
>> +    echo "$(date): $file finished with status=$STATUS" >>$LOGFILE
>> +done
> 
> execute_fsfreeze_hook() will fail the freeze process if this script fails. Two
> comments:
> 
>  1. Do we want to fail the freeze process if one of the sub-scripts fails?
>     If yes, then we have to exit 1 in the first failure

I originally thought the hooks are optional; even they failed, filesystem-level
consistency are still kept in the snapshot.
However, if we are going to fail fsfreeze process by one of sub-scripts'
failure, we also need to notify scripts which already succeeded to pre-freeze
to thaw (or abort) freezing, before exit 1 here.

>  2. The exit status of the script will echo's exit status. I doubt we want that

Do you mean $STATUS (not $status) is specialized in some shell environments?

>> diff --git a/docs/qemu-guest-agent/fsfreeze-hook.d.sample/mysql-flush.sh b/docs/qemu-guest-agent/fsfreeze-hook.d.sample/mysql-flush.sh
>> new file mode 100755
>> index 0000000..e6d7998
>> --- /dev/null
>> +++ b/docs/qemu-guest-agent/fsfreeze-hook.d.sample/mysql-flush.sh
>> @@ -0,0 +1,47 @@
>> +#!/bin/sh
>> +
>> +# Flush MySQL tables to the disk before the filesystem is freezed.
>> +# At the same time, this keeps a read lock while the filesystem is freezed
>> +# in order to avoid write accesses by the other clients.
>> +
>> +MYSQL="mysql -uroot" #"-prootpassword"
>> +FIFO=/tmp/mysql-flush.fifo
>> +
>> +flush_and_wait() {
>> +    printf "FLUSH TABLES WITH READ LOCK \\G\n"
>> +    read < $FIFO
>> +    printf "UNLOCK TABLES \\G\n"
>> +}
>> +
>> +case "$1" in
>> +    freeze)
>> +        mkfifo $FIFO || exit 1
>> +        flush_and_wait | $MYSQL &
> 
> Honest question: what happens if I don't have mysql installed?

Ah OK, availability of mysql should be checked in advance.

Thanks,
Luiz Capitulino - Nov. 26, 2012, 12:40 p.m.
On Mon, 26 Nov 2012 20:49:17 +0900
Tomoki Sekiyama <tomoki.sekiyama.qu@hitachi.com> wrote:

> On 2012/11/23 1:03, Luiz Capitulino wrote:
> > On Thu, 22 Nov 2012 11:15:49 +0900
> > Tomoki Sekiyama <tomoki.sekiyama.qu@hitachi.com> wrote:
> > 
> >> Adds sample hook scripts for --fsfreeze-hook option of qemu-ga.
> >>   - fsfreeze-hook : execute scripts in fsfreeze-hook.d/
> >>   - fsfreeze-hook.d.sample/mysql-flush.sh : quiesce MySQL before snapshot
> <snip>
> >> +# Iterate executables in directory "fsfreeze-hook.d" with the specified args
> >> +[ ! -d "$FSFREEZE_D" ] && exit 1
> >> +for file in "$FSFREEZE_D"/* ; do
> >> +    is_ignored_file "$file" && continue
> >> +    [ -x "$file" ] || continue
> >> +    echo "$(date): execute $file $@" >>$LOGFILE
> >> +    "$file" "$@" >>$LOGFILE 2>&1
> >> +    STATUS=$?
> >> +    echo "$(date): $file finished with status=$STATUS" >>$LOGFILE
> >> +done
> > 
> > execute_fsfreeze_hook() will fail the freeze process if this script fails. Two
> > comments:
> > 
> >  1. Do we want to fail the freeze process if one of the sub-scripts fails?
> >     If yes, then we have to exit 1 in the first failure
> 
> I originally thought the hooks are optional; even they failed, filesystem-level
> consistency are still kept in the snapshot.
> However, if we are going to fail fsfreeze process by one of sub-scripts'
> failure, we also need to notify scripts which already succeeded to pre-freeze
> to thaw (or abort) freezing, before exit 1 here.

Right, which makes things more complex. I vote for doing it the simpler way
for now then, which is to ignore subscripts exit status. But then you should
add exit 0 after the loop to avoid this:

> >  2. The exit status of the script will echo's exit status. I doubt we want that
> 
> Do you mean $STATUS (not $status) is specialized in some shell environments?

No, what I meant is that (afaik) when the script finishes, its return status to
qemu-ga is actually going to be the latest call echo exit status because it's
the last command executed in the loop. If echo fails (say no space) then
the script will fail.

We either, ignore any failures in qemu-ga itself (although we should at least
print a warning there) and/or add exit 0 as the last line of the script.

> 
> >> diff --git a/docs/qemu-guest-agent/fsfreeze-hook.d.sample/mysql-flush.sh b/docs/qemu-guest-agent/fsfreeze-hook.d.sample/mysql-flush.sh
> >> new file mode 100755
> >> index 0000000..e6d7998
> >> --- /dev/null
> >> +++ b/docs/qemu-guest-agent/fsfreeze-hook.d.sample/mysql-flush.sh
> >> @@ -0,0 +1,47 @@
> >> +#!/bin/sh
> >> +
> >> +# Flush MySQL tables to the disk before the filesystem is freezed.
> >> +# At the same time, this keeps a read lock while the filesystem is freezed
> >> +# in order to avoid write accesses by the other clients.
> >> +
> >> +MYSQL="mysql -uroot" #"-prootpassword"
> >> +FIFO=/tmp/mysql-flush.fifo
> >> +
> >> +flush_and_wait() {
> >> +    printf "FLUSH TABLES WITH READ LOCK \\G\n"
> >> +    read < $FIFO
> >> +    printf "UNLOCK TABLES \\G\n"
> >> +}
> >> +
> >> +case "$1" in
> >> +    freeze)
> >> +        mkfifo $FIFO || exit 1
> >> +        flush_and_wait | $MYSQL &
> > 
> > Honest question: what happens if I don't have mysql installed?
> 
> Ah OK, availability of mysql should be checked in advance.
> 
> Thanks,
Tomoki Sekiyama - Nov. 27, 2012, 2:35 a.m.
On 2012/11/26 21:40, Luiz Capitulino wrote:
> On Mon, 26 Nov 2012 20:49:17 +0900
> Tomoki Sekiyama <tomoki.sekiyama.qu@hitachi.com> wrote:
> 
>> On 2012/11/23 1:03, Luiz Capitulino wrote:
>>> On Thu, 22 Nov 2012 11:15:49 +0900
>>> Tomoki Sekiyama <tomoki.sekiyama.qu@hitachi.com> wrote:
>>>
>>>> Adds sample hook scripts for --fsfreeze-hook option of qemu-ga.
>>>>   - fsfreeze-hook : execute scripts in fsfreeze-hook.d/
>>>>   - fsfreeze-hook.d.sample/mysql-flush.sh : quiesce MySQL before snapshot
>> <snip>
>>>> +# Iterate executables in directory "fsfreeze-hook.d" with the specified args
>>>> +[ ! -d "$FSFREEZE_D" ] && exit 1
>>>> +for file in "$FSFREEZE_D"/* ; do
>>>> +    is_ignored_file "$file" && continue
>>>> +    [ -x "$file" ] || continue
>>>> +    echo "$(date): execute $file $@" >>$LOGFILE
>>>> +    "$file" "$@" >>$LOGFILE 2>&1
>>>> +    STATUS=$?
>>>> +    echo "$(date): $file finished with status=$STATUS" >>$LOGFILE
>>>> +done
>>>
>>> execute_fsfreeze_hook() will fail the freeze process if this script fails. Two
>>> comments:
>>>
>>>  1. Do we want to fail the freeze process if one of the sub-scripts fails?
>>>     If yes, then we have to exit 1 in the first failure
>>
>> I originally thought the hooks are optional; even they failed, filesystem-level
>> consistency are still kept in the snapshot.
>> However, if we are going to fail fsfreeze process by one of sub-scripts'
>> failure, we also need to notify scripts which already succeeded to pre-freeze
>> to thaw (or abort) freezing, before exit 1 here.
> 
> Right, which makes things more complex. I vote for doing it the simpler way
> for now then, which is to ignore subscripts exit status. But then you should
> add exit 0 after the loop to avoid this:

I agree this way.

>>>  2. The exit status of the script will echo's exit status. I doubt we want that
>>
>> Do you mean $STATUS (not $status) is specialized in some shell environments?
> 
> No, what I meant is that (afaik) when the script finishes, its return status to
> qemu-ga is actually going to be the latest call echo exit status because it's
> the last command executed in the loop. If echo fails (say no space) then
> the script will fail.
> 
> We either, ignore any failures in qemu-ga itself (although we should at least
> print a warning there) and/or add exit 0 as the last line of the script.

Now I got the point. I will fix this at the next version.

Thanks,

Patch

diff --git a/docs/qemu-guest-agent/fsfreeze-hook b/docs/qemu-guest-agent/fsfreeze-hook
new file mode 100755
index 0000000..319f68c
--- /dev/null
+++ b/docs/qemu-guest-agent/fsfreeze-hook
@@ -0,0 +1,31 @@ 
+#!/bin/sh
+
+# This script is executed when a guest agent receives fsfreeze-freeze and
+# fsfreeze-thaw command, if it is specified in --fsfreeze-hook (-F)
+# option of qemu-ga or placed in default path (/etc/qemu/fsfreeze-hook).
+# When the agent receives fsfreeze-freeze request, this script is issued with
+# "freeze" argument before the filesystem is freezed. And for fsfreeze-thaw
+# request, it is issued with "thaw" argument after filesystem is thawed.
+
+LOGFILE=/var/log/qga-fsfreeze-hook.log
+FSFREEZE_D=$(dirname -- "$0")/fsfreeze-hook.d
+
+# Check whether file $1 is a backup or rpm-generated file and should be ignored
+is_ignored_file() {
+    case "$1" in
+        *~ | *.bak | *.orig | *.rpmnew | *.rpmorig | *.rpmsave)
+            return 0 ;;
+    esac
+    return 1
+}
+
+# Iterate executables in directory "fsfreeze-hook.d" with the specified args
+[ ! -d "$FSFREEZE_D" ] && exit 1
+for file in "$FSFREEZE_D"/* ; do
+    is_ignored_file "$file" && continue
+    [ -x "$file" ] || continue
+    echo "$(date): execute $file $@" >>$LOGFILE
+    "$file" "$@" >>$LOGFILE 2>&1
+    STATUS=$?
+    echo "$(date): $file finished with status=$STATUS" >>$LOGFILE
+done
diff --git a/docs/qemu-guest-agent/fsfreeze-hook.d.sample/mysql-flush.sh b/docs/qemu-guest-agent/fsfreeze-hook.d.sample/mysql-flush.sh
new file mode 100755
index 0000000..e6d7998
--- /dev/null
+++ b/docs/qemu-guest-agent/fsfreeze-hook.d.sample/mysql-flush.sh
@@ -0,0 +1,47 @@ 
+#!/bin/sh
+
+# Flush MySQL tables to the disk before the filesystem is freezed.
+# At the same time, this keeps a read lock while the filesystem is freezed
+# in order to avoid write accesses by the other clients.
+
+MYSQL="mysql -uroot" #"-prootpassword"
+FIFO=/tmp/mysql-flush.fifo
+
+flush_and_wait() {
+    printf "FLUSH TABLES WITH READ LOCK \\G\n"
+    read < $FIFO
+    printf "UNLOCK TABLES \\G\n"
+}
+
+case "$1" in
+    freeze)
+        mkfifo $FIFO || exit 1
+        flush_and_wait | $MYSQL &
+        # wait until every block is flushed
+        while [ "$(echo 'SHOW STATUS LIKE "Key_blocks_not_flushed"' |\
+                 $MYSQL | tail -1 | cut -f 2)" -gt 0 ]; do
+            sleep 1
+        done
+        # for InnoDB, wait until every log is flushed
+        INNODB_STATUS=$(mktemp /tmp/mysql-flush.XXXXXX)
+        [ $? -ne 0 ] && exit 2
+        trap "rm -f $INNODB_STATUS" SIGINT
+        while :; do
+            printf "SHOW ENGINE INNODB STATUS \\G" | $MYSQL > $INNODB_STATUS
+            LOG_CURRENT=$(grep 'Log sequence number' $INNODB_STATUS |\
+                          tr -s ' ' | cut -d' ' -f4)
+            LOG_FLUSHED=$(grep 'Log flushed up to' $INNODB_STATUS |\
+                          tr -s ' ' | cut -d' ' -f5)
+            [ "$LOG_CURRENT" = "$LOG_FLUSHED" ] && break
+            sleep 1
+        done
+        rm -f $INNODB_STATUS
+        ;;
+
+    thaw)
+        [ ! -p $FIFO ] && exit 1
+        echo > $FIFO
+        rm -f $FIFO
+        ;;
+esac
+