Patchwork Download packages from github in tar.gz format instead git format.

login
register
mail settings
Submitter Marek Belisko
Date Sept. 15, 2012, 1:07 p.m.
Message ID <1347714423-13242-1-git-send-email-marek.belisko@open-nandra.com>
Download mbox | patch
Permalink /patch/184079/
State Changes Requested
Headers show

Comments

Marek Belisko - Sept. 15, 2012, 1:07 p.m.
Signed-off-by: Marek Belisko <marek.belisko@open-nandra.com>
---
 package/cpuload/cpuload.mk                       |    2 +-
 package/input-event-daemon/input-event-daemon.mk |    2 +-
 package/jsmin/jsmin.mk                           |    2 +-
 package/libiqrf/libiqrf.mk                       |    2 +-
 package/mtdev2tuio/mtdev2tuio.mk                 |    2 +-
 package/omap-u-boot-utils/omap-u-boot-utils.mk   |    2 +-
 package/socketcand/socketcand.mk                 |    2 +-
 package/ti-utils/ti-utils.mk                     |    2 +-
 package/tslib/tslib.mk                           |    2 +-
 package/yajl/yajl.mk                             |    2 +-
 10 files changed, 10 insertions(+), 10 deletions(-)
Samuel Martin - Sept. 15, 2012, 3:04 p.m.
Nack.

Though github provides a "download" tab, with a button to directly
download the source tarball,
the generated tarball url does contain the beginning of the commit sha1.
So, the download url cannot be easily resolved, at least the current
download infrastructure does not support it.

To, use a fix url stating with "http://github.com/downloads/", the
maintainers of a github project has to upload something.

As ref, a thread also talking about github download urls:
http://lists.busybox.net/pipermail/buildroot/2012-August/057570.html


2012/9/15 Marek Belisko <marek.belisko@open-nandra.com>:
> Signed-off-by: Marek Belisko <marek.belisko@open-nandra.com>
> ---
>  package/cpuload/cpuload.mk                       |    2 +-
>  package/input-event-daemon/input-event-daemon.mk |    2 +-
>  package/jsmin/jsmin.mk                           |    2 +-
>  package/libiqrf/libiqrf.mk                       |    2 +-
>  package/mtdev2tuio/mtdev2tuio.mk                 |    2 +-
>  package/omap-u-boot-utils/omap-u-boot-utils.mk   |    2 +-
>  package/socketcand/socketcand.mk                 |    2 +-
>  package/ti-utils/ti-utils.mk                     |    2 +-
>  package/tslib/tslib.mk                           |    2 +-
>  package/yajl/yajl.mk                             |    2 +-
>  10 files changed, 10 insertions(+), 10 deletions(-)

BTW, in this list, only yajl does upload some material, but the
tarball of the yajl version we use in BR in is not available for
download on the download page... :(

Regards,
Belisko Marek - Sept. 15, 2012, 7:34 p.m.
On Sat, Sep 15, 2012 at 5:04 PM, Samuel Martin <s.martin49@gmail.com> wrote:
> Nack.
>
> Though github provides a "download" tab, with a button to directly
> download the source tarball,
> the generated tarball url does contain the beginning of the commit sha1.
> So, the download url cannot be easily resolved, at least the current
> download infrastructure does not support it.
I was too fast with this patch. I retest few packages and all was working so
realized everything can be converted.

I find a way how to get tar.gz from github but not sure if it will be accepted
when post a patch:

When define in mk file:
RASPBERRYPI_FIRMWARE_VERSION = d29291e3e7d90f0137a1416845c7569cf291255e
RASPBERRYPI_FIRMWARE_SOURCE =
raspberrypi-firmware-$(RASPBERRYPI_FIRMWARE_VERSION).tar.gz
RASPBERRYPI_FIRMWARE_SITE =
http://github.com/raspberrypi/firmware/tarball/master

It start downloading correct git sha1:

>>> raspberrypi-firmware d29291e3e7d90f0137a1416845c7569cf291255e Downloading
--2012-09-15 21:31:37--
http://github.com/raspberrypi/firmware/tarball/master/raspberrypi-firmware-d29291e3e7d90f0137a1416845c7569cf291255e.tar.gz
Resolving github.com (github.com)... 207.97.227.239
Connecting to github.com (github.com)|207.97.227.239|:80... connected.
HTTP request sent, awaiting response... 301 Moved Permanently
Location: https://github.com/raspberrypi/firmware/tarball/master/raspberrypi-firmware-d29291e3e7d90f0137a1416845c7569cf291255e.tar.gz
[following]
--2012-09-15 21:31:37--
https://github.com/raspberrypi/firmware/tarball/master/raspberrypi-firmware-d29291e3e7d90f0137a1416845c7569cf291255e.tar.gz
Connecting to github.com (github.com)|207.97.227.239|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location: https://nodeload.github.com/raspberrypi/firmware/tarball/master
[following]
--2012-09-15 21:31:38--
https://nodeload.github.com/raspberrypi/firmware/tarball/master
Resolving nodeload.github.com (nodeload.github.com)... 207.97.227.252
Connecting to nodeload.github.com
(nodeload.github.com)|207.97.227.252|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 102851634 (98M) [application/x-gzip]
Saving to: `/home/open-nandra/projects/buildroot-new/dl/raspberrypi-firmware-d29291e3e7d90f0137a1416845c7569cf291255e.tar.gz.tmp'

Can I use this method?

>
> To, use a fix url stating with "http://github.com/downloads/", the
> maintainers of a github project has to upload something.
>
> As ref, a thread also talking about github download urls:
> http://lists.busybox.net/pipermail/buildroot/2012-August/057570.html
>
>
> 2012/9/15 Marek Belisko <marek.belisko@open-nandra.com>:
>> Signed-off-by: Marek Belisko <marek.belisko@open-nandra.com>
>> ---
>>  package/cpuload/cpuload.mk                       |    2 +-
>>  package/input-event-daemon/input-event-daemon.mk |    2 +-
>>  package/jsmin/jsmin.mk                           |    2 +-
>>  package/libiqrf/libiqrf.mk                       |    2 +-
>>  package/mtdev2tuio/mtdev2tuio.mk                 |    2 +-
>>  package/omap-u-boot-utils/omap-u-boot-utils.mk   |    2 +-
>>  package/socketcand/socketcand.mk                 |    2 +-
>>  package/ti-utils/ti-utils.mk                     |    2 +-
>>  package/tslib/tslib.mk                           |    2 +-
>>  package/yajl/yajl.mk                             |    2 +-
>>  10 files changed, 10 insertions(+), 10 deletions(-)
>
> BTW, in this list, only yajl does upload some material, but the
> tarball of the yajl version we use in BR in is not available for
> download on the download page... :(
>
> Regards,
>
> --
> Sam

Thanks,

mbe
Samuel Martin - Sept. 15, 2012, 8:32 p.m.
Hi Marek, all,

2012/9/15 Belisko Marek <marek.belisko@gmail.com>:
> On Sat, Sep 15, 2012 at 5:04 PM, Samuel Martin <s.martin49@gmail.com> wrote:
>> Nack.
>>
>> Though github provides a "download" tab, with a button to directly
>> download the source tarball,
>> the generated tarball url does contain the beginning of the commit sha1.
>> So, the download url cannot be easily resolved, at least the current
>> download infrastructure does not support it.
> I was too fast with this patch. I retest few packages and all was working so
> realized everything can be converted.
>
> I find a way how to get tar.gz from github but not sure if it will be accepted
> when post a patch:
>
> When define in mk file:
> RASPBERRYPI_FIRMWARE_VERSION = d29291e3e7d90f0137a1416845c7569cf291255e
> RASPBERRYPI_FIRMWARE_SOURCE =
> raspberrypi-firmware-$(RASPBERRYPI_FIRMWARE_VERSION).tar.gz
> RASPBERRYPI_FIRMWARE_SITE =
> http://github.com/raspberrypi/firmware/tarball/master
>
> It start downloading correct git sha1:
>
>>>> raspberrypi-firmware d29291e3e7d90f0137a1416845c7569cf291255e Downloading
> --2012-09-15 21:31:37--
> http://github.com/raspberrypi/firmware/tarball/master/raspberrypi-firmware-d29291e3e7d90f0137a1416845c7569cf291255e.tar.gz
> Resolving github.com (github.com)... 207.97.227.239
> Connecting to github.com (github.com)|207.97.227.239|:80... connected.
> HTTP request sent, awaiting response... 301 Moved Permanently
> Location: https://github.com/raspberrypi/firmware/tarball/master/raspberrypi-firmware-d29291e3e7d90f0137a1416845c7569cf291255e.tar.gz
> [following]
> --2012-09-15 21:31:37--
> https://github.com/raspberrypi/firmware/tarball/master/raspberrypi-firmware-d29291e3e7d90f0137a1416845c7569cf291255e.tar.gz
> Connecting to github.com (github.com)|207.97.227.239|:443... connected.
> HTTP request sent, awaiting response... 302 Found
> Location: https://nodeload.github.com/raspberrypi/firmware/tarball/master
> [following]
> --2012-09-15 21:31:38--
> https://nodeload.github.com/raspberrypi/firmware/tarball/master
> Resolving nodeload.github.com (nodeload.github.com)... 207.97.227.252
> Connecting to nodeload.github.com
> (nodeload.github.com)|207.97.227.252|:443... connected.
> HTTP request sent, awaiting response... 200 OK
> Length: 102851634 (98M) [application/x-gzip]
> Saving to: `/home/open-nandra/projects/buildroot-new/dl/raspberrypi-firmware-d29291e3e7d90f0137a1416845c7569cf291255e.tar.gz.tmp'
>
> Can I use this method?

Of course you can! :-)


BTW, quick tests show that the beginning of the sha1 is enough for
github to match the revision you want to download, as well as
prefixing the sha1 with 'g'.
So, for example, it could be:
RASPBERRYPI_FIRMWARE_VERSION = d29291e
RASPBERRYPI_FIRMWARE_SOURCE =
raspberrypi-firmware-g$(RASPBERRYPI_FIRMWARE_VERSION).tar.gz
or:
RASPBERRYPI_FIRMWARE_VERSION = d29291e
RASPBERRYPI_FIRMWARE_SOURCE =
raspberrypi-firmware-$(RASPBERRYPI_FIRMWARE_VERSION).tar.gz

IMO, this is the way to go for non-tagged version.

For tagged version, the following seems working fine (to be confirmed):
CPULOAD_VERSION = v0.3
CPULOAD_SOURCE = kelvincheung-cpuload-$(CPULOAD_VERSION).tar.gz
CPULOAD_SITE = http://github.com/kelvincheung/cpuload/$(CPULOAD_VERSION)

If it is confirmed that this works, I'll definitely believe that's the
way to go ;-)

Care to give a try and resend a proper patch?
And, don't hesitate to add some note in the doc about how to form
*_SITE and *_SOURCE variables for github hosted projects.


Cheers,
Belisko Marek - Sept. 15, 2012, 8:48 p.m.
Hi Samuel,

On Sat, Sep 15, 2012 at 10:32 PM, Samuel Martin <s.martin49@gmail.com> wrote:
> Hi Marek, all,
>
> 2012/9/15 Belisko Marek <marek.belisko@gmail.com>:
>> On Sat, Sep 15, 2012 at 5:04 PM, Samuel Martin <s.martin49@gmail.com> wrote:
>>> Nack.
>>>
>>> Though github provides a "download" tab, with a button to directly
>>> download the source tarball,
>>> the generated tarball url does contain the beginning of the commit sha1.
>>> So, the download url cannot be easily resolved, at least the current
>>> download infrastructure does not support it.
>> I was too fast with this patch. I retest few packages and all was working so
>> realized everything can be converted.
>>
>> I find a way how to get tar.gz from github but not sure if it will be accepted
>> when post a patch:
>>
>> When define in mk file:
>> RASPBERRYPI_FIRMWARE_VERSION = d29291e3e7d90f0137a1416845c7569cf291255e
>> RASPBERRYPI_FIRMWARE_SOURCE =
>> raspberrypi-firmware-$(RASPBERRYPI_FIRMWARE_VERSION).tar.gz
>> RASPBERRYPI_FIRMWARE_SITE =
>> http://github.com/raspberrypi/firmware/tarball/master
>>
>> It start downloading correct git sha1:
>>
>>>>> raspberrypi-firmware d29291e3e7d90f0137a1416845c7569cf291255e Downloading
>> --2012-09-15 21:31:37--
>> http://github.com/raspberrypi/firmware/tarball/master/raspberrypi-firmware-d29291e3e7d90f0137a1416845c7569cf291255e.tar.gz
>> Resolving github.com (github.com)... 207.97.227.239
>> Connecting to github.com (github.com)|207.97.227.239|:80... connected.
>> HTTP request sent, awaiting response... 301 Moved Permanently
>> Location: https://github.com/raspberrypi/firmware/tarball/master/raspberrypi-firmware-d29291e3e7d90f0137a1416845c7569cf291255e.tar.gz
>> [following]
>> --2012-09-15 21:31:37--
>> https://github.com/raspberrypi/firmware/tarball/master/raspberrypi-firmware-d29291e3e7d90f0137a1416845c7569cf291255e.tar.gz
>> Connecting to github.com (github.com)|207.97.227.239|:443... connected.
>> HTTP request sent, awaiting response... 302 Found
>> Location: https://nodeload.github.com/raspberrypi/firmware/tarball/master
>> [following]
>> --2012-09-15 21:31:38--
>> https://nodeload.github.com/raspberrypi/firmware/tarball/master
>> Resolving nodeload.github.com (nodeload.github.com)... 207.97.227.252
>> Connecting to nodeload.github.com
>> (nodeload.github.com)|207.97.227.252|:443... connected.
>> HTTP request sent, awaiting response... 200 OK
>> Length: 102851634 (98M) [application/x-gzip]
>> Saving to: `/home/open-nandra/projects/buildroot-new/dl/raspberrypi-firmware-d29291e3e7d90f0137a1416845c7569cf291255e.tar.gz.tmp'
>>
>> Can I use this method?
>
> Of course you can! :-)
>
>
> BTW, quick tests show that the beginning of the sha1 is enough for
> github to match the revision you want to download, as well as
> prefixing the sha1 with 'g'.
> So, for example, it could be:
> RASPBERRYPI_FIRMWARE_VERSION = d29291e
> RASPBERRYPI_FIRMWARE_SOURCE =
> raspberrypi-firmware-g$(RASPBERRYPI_FIRMWARE_VERSION).tar.gz
> or:
> RASPBERRYPI_FIRMWARE_VERSION = d29291e
> RASPBERRYPI_FIRMWARE_SOURCE =
> raspberrypi-firmware-$(RASPBERRYPI_FIRMWARE_VERSION).tar.gz
Yes I confirmed this method also work.
>
> IMO, this is the way to go for non-tagged version.
>
> For tagged version, the following seems working fine (to be confirmed):
> CPULOAD_VERSION = v0.3
> CPULOAD_SOURCE = kelvincheung-cpuload-$(CPULOAD_VERSION).tar.gz
> CPULOAD_SITE = http://github.com/kelvincheung/cpuload/$(CPULOAD_VERSION)
This will work but this way:
 CPULOAD_SITE =
http://github.com/kelvincheung/cpuload/tarball/$(CPULOAD_VERSION)
(same is used in package package/sqlcipher/sqlcipher.mk)
>
> If it is confirmed that this works, I'll definitely believe that's the
> way to go ;-)
>
> Care to give a try and resend a proper patch?
> And, don't hesitate to add some note in the doc about how to form
> *_SITE and *_SOURCE variables for github hosted projects.
Sure. Will do.
>
>
> Cheers,
>
> --
> Sam

Thanks,

mbe

Patch

diff --git a/package/cpuload/cpuload.mk b/package/cpuload/cpuload.mk
index d25c504..6bbe969 100644
--- a/package/cpuload/cpuload.mk
+++ b/package/cpuload/cpuload.mk
@@ -5,7 +5,7 @@ 
 #############################################################
 
 CPULOAD_VERSION = v0.3
-CPULOAD_SITE = git://github.com/kelvincheung/cpuload.git
+CPULOAD_SITE = http://github.com/downloads/kelvincheung/cpuload
 CPULOAD_LICENSE = GPLv2
 CPULOAD_LICENSE_FILES = COPYING
 
diff --git a/package/input-event-daemon/input-event-daemon.mk b/package/input-event-daemon/input-event-daemon.mk
index f99bfbe..c1be637 100644
--- a/package/input-event-daemon/input-event-daemon.mk
+++ b/package/input-event-daemon/input-event-daemon.mk
@@ -5,7 +5,7 @@ 
 #############################################################
 
 INPUT_EVENT_DAEMON_VERSION = v0.1.3
-INPUT_EVENT_DAEMON_SITE = git://github.com/gandro/input-event-daemon.git
+INPUT_EVENT_DAEMON_SITE = http://github.com/downloads/gandro/input-event-daemon
 INPUT_EVENT_DAEMON_LICENSE = input-event-daemon license
 INPUT_EVENT_DAEMON_LICENSE_FILES = README
 
diff --git a/package/jsmin/jsmin.mk b/package/jsmin/jsmin.mk
index fb8c214..d7a9f76 100644
--- a/package/jsmin/jsmin.mk
+++ b/package/jsmin/jsmin.mk
@@ -1,5 +1,5 @@ 
 JSMIN_VERSION = a9b47554d5684a55301a2eb7ca8480b7ee7630d4
-JSMIN_SITE = git://github.com/douglascrockford/JSMin.git
+JSMIN_SITE = http://github.com/downloads/douglascrockford/JSMin
 
 define JSMIN_BUILD_CMDS
 	$(TARGET_CONFIGURE_OPTS) $(MAKE) -C $(@D) jsmin
diff --git a/package/libiqrf/libiqrf.mk b/package/libiqrf/libiqrf.mk
index 4cf1541..59dffa2 100644
--- a/package/libiqrf/libiqrf.mk
+++ b/package/libiqrf/libiqrf.mk
@@ -5,7 +5,7 @@ 
 #############################################################
 
 LIBIQRF_VERSION = v0.1.0
-LIBIQRF_SITE = git://github.com/nandra/libiqrf.git
+LIBIQRF_SITE = http://github.com/downloads/nandra/libiqrf
 LIBIQRF_INSTALL_STAGING = YES
 
 LIBIQRF_DEPENDENCIES = libusb
diff --git a/package/mtdev2tuio/mtdev2tuio.mk b/package/mtdev2tuio/mtdev2tuio.mk
index a86c0bf..e93e3fc 100644
--- a/package/mtdev2tuio/mtdev2tuio.mk
+++ b/package/mtdev2tuio/mtdev2tuio.mk
@@ -4,7 +4,7 @@ 
 #
 #############################################################
 MTDEV2TUIO_VERSION = e1e7378d86
-MTDEV2TUIO_SITE = git://github.com/olivopaolo/mtdev2tuio.git
+MTDEV2TUIO_SITE = http://github.com/downloads/olivopaolo/mtdev2tuio
 MTDEV2TUIO_DEPENDENCIES = mtdev liblo
 MTDEV2TUIO_LICENSE = GPLv3+
 MTDEV2TUIO_LICENSE_FILES = COPYING
diff --git a/package/omap-u-boot-utils/omap-u-boot-utils.mk b/package/omap-u-boot-utils/omap-u-boot-utils.mk
index 8405f62..51508de 100644
--- a/package/omap-u-boot-utils/omap-u-boot-utils.mk
+++ b/package/omap-u-boot-utils/omap-u-boot-utils.mk
@@ -5,7 +5,7 @@ 
 #############################################################
 
 OMAP_U_BOOT_UTILS_VERSION = 8aff852322
-OMAP_U_BOOT_UTILS_SITE = http://github.com/nmenon/omap-u-boot-utils.git
+OMAP_U_BOOT_UTILS_SITE = http://github.com/downloads/nmenon/omap-u-boot-utils
 OMAP_U_BOOT_UTILS_SITE_METHOD = git
 
 define HOST_OMAP_U_BOOT_UTILS_BUILD_CMDS
diff --git a/package/socketcand/socketcand.mk b/package/socketcand/socketcand.mk
index a1e672f..ff65d85 100644
--- a/package/socketcand/socketcand.mk
+++ b/package/socketcand/socketcand.mk
@@ -4,7 +4,7 @@ 
 #
 #############################################################
 SOCKETCAND_VERSION = 7d06986fa4b5fd2c210ec4e248dab41107be1ccd
-SOCKETCAND_SITE = git://github.com/dschanoeh/socketcand.git
+SOCKETCAND_SITE = http://github.com/downloads/dschanoeh/socketcand
 SOCKETCAND_AUTORECONF = YES
 SOCKETCAND_DEPENDENCIES = libconfig
 
diff --git a/package/ti-utils/ti-utils.mk b/package/ti-utils/ti-utils.mk
index ac54370..4f8b320 100644
--- a/package/ti-utils/ti-utils.mk
+++ b/package/ti-utils/ti-utils.mk
@@ -5,7 +5,7 @@ 
 #############################################################
 
 TI_UTILS_VERSION = 06dbdb2727354b5f3ad7c723897f40051fddee49
-TI_UTILS_SITE = git://github.com/gxk/ti-utils.git
+TI_UTILS_SITE = http://github.com/downloads/gxk/ti-utils
 TI_UTILS_DEPENDENCIES = libnl
 
 define TI_UTILS_BUILD_CMDS
diff --git a/package/tslib/tslib.mk b/package/tslib/tslib.mk
index 8f30c25..1650bd7 100644
--- a/package/tslib/tslib.mk
+++ b/package/tslib/tslib.mk
@@ -4,7 +4,7 @@ 
 #
 #############################################################
 TSLIB_VERSION = 412d99d8b92c
-TSLIB_SITE = git://github.com/kergoth/tslib.git
+TSLIB_SITE = http://github.com/downloads/kergoth/tslib
 TSLIB_AUTORECONF = YES
 TSLIB_INSTALL_STAGING = YES
 TSLIB_INSTALL_STAGING_OPT = DESTDIR=$(STAGING_DIR) LDFLAGS=-L$(STAGING_DIR)/usr/lib install
diff --git a/package/yajl/yajl.mk b/package/yajl/yajl.mk
index 9cebeb7..b1d7fca 100644
--- a/package/yajl/yajl.mk
+++ b/package/yajl/yajl.mk
@@ -5,7 +5,7 @@ 
 ################################################################################
 
 YAJL_VERSION = 2.0.4
-YAJL_SITE = git://github.com/lloyd/yajl.git
+YAJL_SITE = http://github.com/downloads/lloyd/yaju
 YAJL_INSTALL_STAGING = YES
 YAJL_LICENSE = ISC
 YAJL_LICENSE_FILES = COPYING