Patchwork infra: Add a way to apply patches using git

login
register
mail settings
Submitter Jérôme Pouiller
Date July 2, 2014, 5:20 p.m.
Message ID <1404321605-20073-1-git-send-email-jezz@sysmic.org>
Download mbox | patch
Permalink /patch/366435/
State Rejected
Headers show

Comments

Jérôme Pouiller - July 2, 2014, 5:20 p.m.
This option is mainly used for package developpers. When activated,
Buildroot will automaticaly create a git repository for each package
it extract. Initial version is tagged as `upstream'. Next, during
patch step, git is used instead of patch. BTW, patches are easier
to modify/fix/rebase/cherry-pick/upstream. Once satisfied with
changes, developper may use
    git format-patch -o $TOPDIR/package/$PKG upstream
to save its patches to Buildroot.

Notice this option should be compatible with `guilt' (a clone of
`quilt')

Notice you can also use this feature for a specific package using
    make BR2_USE_GIT_TO_PATCH=y $PKG-dirclean $PKG-patch

Current known limitations:
  - If a patch cannot be handled by `git am', we use `git apply`. In
    this case, patch description is not reported in commit message.
  - When patch is handled with `git apply`, subject used may not be
    perfect (it is extracted from patch filename)
  - `git format-patch' does not prefix patch files with name of package.
  - If upstream was downloaded with git, we may also run
    `git remote add $($(PKG)_SITE)' during git repository creation

Signed-off-by: Jérôme Pouiller <jezz@sysmic.org>
---
 Config.in                        | 18 ++++++++++++++++++
 Makefile                         |  4 ++++
 package/pkg-generic.mk           |  7 +++++++
 support/scripts/apply-patches.sh | 29 +++++++++++++++++++++++++++--
 4 files changed, 56 insertions(+), 2 deletions(-)
Thomas Petazzoni - Nov. 1, 2014, 9:47 p.m.
Dear Jérôme Pouiller,

On Wed,  2 Jul 2014 19:20:05 +0200, Jérôme Pouiller wrote:
> This option is mainly used for package developpers. When activated,
> Buildroot will automaticaly create a git repository for each package
> it extract. Initial version is tagged as `upstream'. Next, during
> patch step, git is used instead of patch. BTW, patches are easier
> to modify/fix/rebase/cherry-pick/upstream. Once satisfied with
> changes, developper may use
>     git format-patch -o $TOPDIR/package/$PKG upstream
> to save its patches to Buildroot.
> 
> Notice this option should be compatible with `guilt' (a clone of
> `quilt')
> 
> Notice you can also use this feature for a specific package using
>     make BR2_USE_GIT_TO_PATCH=y $PKG-dirclean $PKG-patch
> 
> Current known limitations:
>   - If a patch cannot be handled by `git am', we use `git apply`. In
>     this case, patch description is not reported in commit message.
>   - When patch is handled with `git apply`, subject used may not be
>     perfect (it is extracted from patch filename)
>   - `git format-patch' does not prefix patch files with name of package.
>   - If upstream was downloaded with git, we may also run
>     `git remote add $($(PKG)_SITE)' during git repository creation
> 
> Signed-off-by: Jérôme Pouiller <jezz@sysmic.org>

We discussed this proposal at the last Buildroot Developers meeting.
Even though we understand the idea and some of us found it potentially
interesting, we didn't really like the implementation, but nobody
really found a way of implementing something like this in a nice way.

One of the problem is that not all our patches are Git patches, which
you had to work-around in your patch in a way that isn't really nice.

Maybe Arnout and Yann (in Cc) remember more of the issues that we
discussed at the meeting regarding this proposal.

Thanks,

Thomas
Arnout Vandecappelle - Nov. 1, 2014, 10:06 p.m.
On 01/11/14 22:47, Thomas Petazzoni wrote:
> Dear Jérôme Pouiller,
> 
> On Wed,  2 Jul 2014 19:20:05 +0200, Jérôme Pouiller wrote:
>> This option is mainly used for package developpers. When activated,
>> Buildroot will automaticaly create a git repository for each package
>> it extract. Initial version is tagged as `upstream'. Next, during
>> patch step, git is used instead of patch. BTW, patches are easier
>> to modify/fix/rebase/cherry-pick/upstream. Once satisfied with
>> changes, developper may use
>>     git format-patch -o $TOPDIR/package/$PKG upstream
>> to save its patches to Buildroot.
>>
>> Notice this option should be compatible with `guilt' (a clone of
>> `quilt')
>>
>> Notice you can also use this feature for a specific package using
>>     make BR2_USE_GIT_TO_PATCH=y $PKG-dirclean $PKG-patch
>>
>> Current known limitations:
>>   - If a patch cannot be handled by `git am', we use `git apply`. In
>>     this case, patch description is not reported in commit message.
>>   - When patch is handled with `git apply`, subject used may not be
>>     perfect (it is extracted from patch filename)
>>   - `git format-patch' does not prefix patch files with name of package.
>>   - If upstream was downloaded with git, we may also run
>>     `git remote add $($(PKG)_SITE)' during git repository creation
>>
>> Signed-off-by: Jérôme Pouiller <jezz@sysmic.org>
> 
> We discussed this proposal at the last Buildroot Developers meeting.
> Even though we understand the idea and some of us found it potentially
> interesting, we didn't really like the implementation, but nobody
> really found a way of implementing something like this in a nice way.
> 
> One of the problem is that not all our patches are Git patches, which
> you had to work-around in your patch in a way that isn't really nice.
> 
> Maybe Arnout and Yann (in Cc) remember more of the issues that we
> discussed at the meeting regarding this proposal.

 For simple patches, a simple diff is normally sufficient. For more complicated
stuff, it is really worthwhile to clone the upstream repository and work from
there - you'll anyway need to do that to upstream the patches.

 So we felt that the need for this feature did not outweigh the complexity of
its implementation.

 Regards,
 Arnout
Thomas Petazzoni - Nov. 1, 2014, 10:11 p.m.
Dear Arnout Vandecappelle,

On Sat, 01 Nov 2014 23:06:23 +0100, Arnout Vandecappelle wrote:

> > We discussed this proposal at the last Buildroot Developers meeting.
> > Even though we understand the idea and some of us found it potentially
> > interesting, we didn't really like the implementation, but nobody
> > really found a way of implementing something like this in a nice way.
> > 
> > One of the problem is that not all our patches are Git patches, which
> > you had to work-around in your patch in a way that isn't really nice.
> > 
> > Maybe Arnout and Yann (in Cc) remember more of the issues that we
> > discussed at the meeting regarding this proposal.
> 
>  For simple patches, a simple diff is normally sufficient. For more complicated
> stuff, it is really worthwhile to clone the upstream repository and work from
> there - you'll anyway need to do that to upstream the patches.
> 
>  So we felt that the need for this feature did not outweigh the complexity of
> its implementation.

Ah yes, right: we thought that having a Git repository that isn't the
one from upstream made it unpractical to submit patches back to
upstream, and was making the entire feature a bit useless.

Thomas
Jérôme Pouiller - Nov. 3, 2014, 3:01 p.m.
On Saturday 01 November 2014 23:11:25 Thomas Petazzoni wrote:
> Dear Arnout Vandecappelle,
> 
> On Sat, 01 Nov 2014 23:06:23 +0100, Arnout Vandecappelle wrote:
> > > We discussed this proposal at the last Buildroot Developers meeting.
> > > Even though we understand the idea and some of us found it potentially
> > > interesting, we didn't really like the implementation, but nobody
> > > really found a way of implementing something like this in a nice way.
> > > 
> > > One of the problem is that not all our patches are Git patches, which
> > > you had to work-around in your patch in a way that isn't really nice.
> > > 
> > > Maybe Arnout and Yann (in Cc) remember more of the issues that we
> > > discussed at the meeting regarding this proposal.
> >  
> >  For simple patches, a simple diff is normally sufficient. For more
> >  complicated> 
> > stuff, it is really worthwhile to clone the upstream repository and work
> > from there - you'll anyway need to do that to upstream the patches.
> > 
> >  So we felt that the need for this feature did not outweigh the complexity
> >  of> 
> > its implementation.
>
> Ah yes, right: we thought that having a Git repository that isn't the
> one from upstream made it unpractical to submit patches back to
> upstream, and was making the entire feature a bit useless.
I also worried about this restriction. My first idea was to recreate the link 
between source directory and upstream repository with something like:

   cd $PKG_BUILD_DIR
   git clone $PKG_SITE --bare .git
   git config --bool core.bare false
   git rev-parse $PKG_VERSION > .git/HEAD
   git reset

However, my patch began to smell black magic. 


Ideally, I wanted to create an external tool to do this. However, it is not 
easy to integrate an external tool with Makefile system (For example, I did 
not found any handy way to get variables associated with package).


If anyone have ideas about this subject, I would be happy to work on and 
resubmit a patch.
Arnout Vandecappelle - Nov. 3, 2014, 8:21 p.m.
On 03/11/14 16:01, Jérôme Pouiller wrote:
> On Saturday 01 November 2014 23:11:25 Thomas Petazzoni wrote:
>> Dear Arnout Vandecappelle,
>>
>> On Sat, 01 Nov 2014 23:06:23 +0100, Arnout Vandecappelle wrote:
>>>> We discussed this proposal at the last Buildroot Developers meeting.
>>>> Even though we understand the idea and some of us found it potentially
>>>> interesting, we didn't really like the implementation, but nobody
>>>> really found a way of implementing something like this in a nice way.
>>>>
>>>> One of the problem is that not all our patches are Git patches, which
>>>> you had to work-around in your patch in a way that isn't really nice.
>>>>
>>>> Maybe Arnout and Yann (in Cc) remember more of the issues that we
>>>> discussed at the meeting regarding this proposal.
>>>  
>>>  For simple patches, a simple diff is normally sufficient. For more
>>>  complicated> 
>>> stuff, it is really worthwhile to clone the upstream repository and work
>>> from there - you'll anyway need to do that to upstream the patches.
>>>
>>>  So we felt that the need for this feature did not outweigh the complexity
>>>  of> 
>>> its implementation.
>>
>> Ah yes, right: we thought that having a Git repository that isn't the
>> one from upstream made it unpractical to submit patches back to
>> upstream, and was making the entire feature a bit useless.
> I also worried about this restriction. My first idea was to recreate the link 
> between source directory and upstream repository with something like:
> 
>    cd $PKG_BUILD_DIR
>    git clone $PKG_SITE --bare .git

 This would only work if $PKG_SITE happened to be a git repository... But since
we prefer to use release tarballs, the usual situation should be that the
repository is somewhere else.

>    git config --bool core.bare false
>    git rev-parse $PKG_VERSION > .git/HEAD
>    git reset
> 
> However, my patch began to smell black magic. 
> 
> 
> Ideally, I wanted to create an external tool to do this. However, it is not 
> easy to integrate an external tool with Makefile system (For example, I did 
> not found any handy way to get variables associated with package).

 At the BR developer meeting, we had the feeling that the number of situations
where such a tool is useful is very limited, so that it is really not worth the
effort to invest in it.


 Regards,
 Arnout

> 
> 
> If anyone have ideas about this subject, I would be happy to work on and 
> resubmit a patch.
> 
>

Patch

diff --git a/Config.in b/Config.in
index 50968fb..3645fe4 100644
--- a/Config.in
+++ b/Config.in
@@ -541,6 +541,24 @@  config BR2_GLOBAL_PATCH_DIR
 	  Otherwise, if the directory <global-patch-dir>/<packagename> exists,
 	  then all *.patch files in the directory will be applied.
 
+config BR2_USE_GIT_TO_PATCH
+	bool "use git repositories to handle patches"
+	help
+	  This option is mainly used for package developpers. When activated,
+	  Buildroot will automaticaly create a git repository for each package
+	  it extract. Initial version is tagged as `upstream'. Next, during
+	  patch step, git is used instead of patch. BTW, patches are easier
+	  to modify/fix/rebase/cherry-pick/upstream. Once satisfied with 
+	  changes, developper may use
+	      git format-patch -o $TOPDIR/package/$PKG upstream
+	  to save its patches to Buildroot.
+
+	  Notice this option should be compatible with `guilt' (a clone of
+	  `quilt')
+
+	  Notice you can also use this feature for a specific package using
+	      make BR2_USE_GIT_TO_PATCH=y $PKG-dirclean $PKG-patch
+
 endmenu
 
 source "toolchain/Config.in"
diff --git a/Makefile b/Makefile
index b3bd91e..b35e5a6 100644
--- a/Makefile
+++ b/Makefile
@@ -177,6 +177,10 @@  ifeq ($(filter $(noconfig_targets),$(MAKECMDGOALS)),)
 -include $(BR2_CONFIG)
 endif
 
+# apply-patches.sh rely on this varaible to know git should be used instead 
+# of patch
+export BR2_USE_GIT_TO_PATCH
+
 # To put more focus on warnings, be less verbose as default
 # Use 'make V=1' to see the full commands
 ifdef V
diff --git a/package/pkg-generic.mk b/package/pkg-generic.mk
index ab1f8d2..86368cc 100644
--- a/package/pkg-generic.mk
+++ b/package/pkg-generic.mk
@@ -108,6 +108,13 @@  $(BUILD_DIR)/%/.stamp_extracted:
 # some packages have messed up permissions inside
 	$(Q)chmod -R +rw $(@D)
 	$(foreach hook,$($(PKG)_POST_EXTRACT_HOOKS),$(call $(hook))$(sep))
+ifeq ($(BR2_USE_GIT_TO_PATCH),y)
+	$(Q)cd $(@D) ; \
+	git init ; \
+	git add . > /dev/null ; \
+	git commit -m "$($(PKG)_RAWNAME) $($(PKG)_VERSION)" > /dev/null ; \
+	git tag upstream
+endif
 	$(Q)touch $@
 	@$(call step_end,extract)
 
diff --git a/support/scripts/apply-patches.sh b/support/scripts/apply-patches.sh
index 37f2d81..0524b9a 100755
--- a/support/scripts/apply-patches.sh
+++ b/support/scripts/apply-patches.sh
@@ -84,8 +84,28 @@  function apply_patch {
 	exit 1
     fi
     echo $patch >> ${builddir}/.applied_patches_list
-    ${uncomp} "${path}/$patch" | patch -g0 -p1 -E -d "${builddir}" -t -N
-    if [ $? != 0 ] ; then
+    if [[ -n "$BR2_USE_GIT_TO_PATCH" && -e "${builddir}/.git" ]]; then
+        abspatch=$(readlink -f "${path}/$patch")
+        pushd ${builddir} > /dev/null
+        ${uncomp} "$abspatch" | git am --reject > /dev/null
+        RET=$?
+        if [[ $RET != 0 ]]; then
+            ${uncomp} "$abspatch" | git apply --index --reject - > /dev/null
+            RET=$?
+            if [[ $RET == 0 ]]; then
+                # This regexp is far to be perfect
+                pkgname=$(echo ${builddir} | sed 's/^.*\///; s/-[0-9\.]\+$//;')
+                msg="$(echo ${patch} | sed "s/^.*\///; s/^$pkgname-//;  s/^[0-9\.]\+-//; s/\.[a-zA-Z]*\$//; y/-/ /;")"
+                git commit -m "$msg" > /dev/null
+                RET=$?
+            fi
+        fi
+        popd > /dev/null
+    else
+        ${uncomp} "${path}/$patch" | patch -g0 -p1 -E -d "${builddir}" -t -N
+        RET=$?
+    fi
+    if [ $RET != 0 ] ; then
         echo "Patch failed!  Please fix ${patch}!"
 	exit 1
     fi
@@ -121,6 +141,11 @@  function scan_patchdir {
 
 scan_patchdir "$patchdir" "$patchpattern"
 
+if [[ -n "$BR2_USE_GIT_TO_PATCH" && ! -e "${builddir}/.git" ]]; then
+    echo "BR2_USE_GIT_TO_PATCH supplied but git repository not found."
+    echo "Falling back to classical patch method."
+fi
+
 # Check for rejects...
 if [ "`find $builddir/ '(' -name '*.rej' -o -name '.*.rej' ')' -print`" ] ; then
     echo "Aborting.  Reject files found."