diff mbox

[LEDE-DEV,2/2] Add script to build common platforms.

Message ID 1464307053-20531-2-git-send-email-greearb@candelatech.com
State Superseded
Headers show

Commit Message

Ben Greear May 26, 2016, 11:57 p.m. UTC
From: Ben Greear <greearb@candelatech.com>

The idea is to be able to allow newbies to easily build images
for common hardware.  These images should be user-friendly, including
luci and other tools that may aid debugging and use of the platform.

Signed-off-by: Ben Greear <greearb@candelatech.com>
---
 buildme.sh                            | 76 +++++++++++++++++++++++++++++++++++
 buildme_targets/x86_64/diffconfig.txt | 26 ++++++++++++
 2 files changed, 102 insertions(+)
 create mode 100755 buildme.sh
 create mode 100644 buildme_targets/x86_64/diffconfig.txt

Comments

Ben Greear May 27, 2016, 1:13 a.m. UTC | #1
On 05/26/2016 04:57 PM, greearb@candelatech.com wrote:
> From: Ben Greear <greearb@candelatech.com>
>
> The idea is to be able to allow newbies to easily build images
> for common hardware.  These images should be user-friendly, including
> luci and other tools that may aid debugging and use of the platform.

Please don't apply this quite yet.  Luci is not running for whatever
reason so I'm still missing something...

Thanks,
Ben
Karl Palsson May 27, 2016, 9:46 a.m. UTC | #2
greearb@candelatech.com wrote:
> From: Ben Greear <greearb@candelatech.com>
> 
> The idea is to be able to allow newbies to easily build images
> for common hardware. These images should be user-friendly,
> including luci and other tools that may aid debugging and use
> of the platform.
> 
> Signed-off-by: Ben Greear <greearb@candelatech.com>
> ---
>  buildme.sh                            | 76 +++++++++++++++++++++++++++++++++++
>  buildme_targets/x86_64/diffconfig.txt | 26 ++++++++++++
>  2 files changed, 102 insertions(+)
>  create mode 100755 buildme.sh
>  create mode 100644 buildme_targets/x86_64/diffconfig.txt
> 


While I've written similar scripts for helping colleagues
bootstrap a build, I'm not entirely convinced that adding more
piles of shell script to the build tree is necessarily an
improvement, if the goal is helping people build an image that
suits them.

Perhaps expanding the README to include examples of some common
tasks, more guidance on how to use menuconfig, more examples of
how to use the feeds scripts, and what they are. Perhaps even, if
you're targetting total beginners, documenting how the image
builder thing works would actually meet your goals better. I've
never understood the image builder myself, it's documented here:
https://wiki.openwrt.org/doc/howto/obtain.firmware.generate but
seems complicated to follow that page.

lede, like openwrt before it, still has large piles of
undocumented usage methods, only uncovered by digging through
script after script. Adding more scripts to hide the
complexity/flexibility/power of some of those other scripts
doesn't seem like a net gain.

Sincerely,
Karl Palsson
Ben Greear May 27, 2016, 1:45 p.m. UTC | #3
On 05/27/2016 02:46 AM, Karl Palsson wrote:
>
> greearb@candelatech.com wrote:
>> From: Ben Greear <greearb@candelatech.com>
>>
>> The idea is to be able to allow newbies to easily build images
>> for common hardware. These images should be user-friendly,
>> including luci and other tools that may aid debugging and use
>> of the platform.
>>
>> Signed-off-by: Ben Greear <greearb@candelatech.com>
>> ---
>>   buildme.sh                            | 76 +++++++++++++++++++++++++++++++++++
>>   buildme_targets/x86_64/diffconfig.txt | 26 ++++++++++++
>>   2 files changed, 102 insertions(+)
>>   create mode 100755 buildme.sh
>>   create mode 100644 buildme_targets/x86_64/diffconfig.txt
>>
>
>
> While I've written similar scripts for helping colleagues
> bootstrap a build, I'm not entirely convinced that adding more
> piles of shell script to the build tree is necessarily an
> improvement, if the goal is helping people build an image that
> suits them.

My goal is more to have an easy way to build a useful image
that should work for most people.  Lots of people and organizations
have their own forks and scripts, and it seems much of the changes are just
tweaks how to the core project is built.  If we can bring some
of this into the main tree maybe that will help everyone work
closer to upstream code.

Extending the README would help, but I don't think a beginner should
have to go through menuconfig at all.  It can be overwhelming even for
those of us who have seen similar things before.  (Try to select 'igb'
as 'y' for instance, it takes manually enabling multiple obscure other
features before it will even work).

> lede, like openwrt before it, still has large piles of
> undocumented usage methods, only uncovered by digging through
> script after script. Adding more scripts to hide the
> complexity/flexibility/power of some of those other scripts
> doesn't seem like a net gain.

A well commented buildme.sh would be a good place to start understanding
the build process.  At least for me, just finding the other scripts is
often the hardest part...so if buildme.sh is adding a package with a feeds
cmd, then that is a very useful piece of knowledge.

Thanks,
Ben
David Lang May 28, 2016, 6:10 a.m. UTC | #4
On Fri, 27 May 2016, Ben Greear wrote:

> On 05/27/2016 02:46 AM, Karl Palsson wrote:
>> 
>> greearb@candelatech.com wrote:
>>> From: Ben Greear <greearb@candelatech.com>
>>> 
>>> The idea is to be able to allow newbies to easily build images
>>> for common hardware. These images should be user-friendly,
>>> including luci and other tools that may aid debugging and use
>>> of the platform.
>>> 
>>> Signed-off-by: Ben Greear <greearb@candelatech.com>
>>> ---
>>>   buildme.sh                            | 76 
>>> +++++++++++++++++++++++++++++++++++
>>>   buildme_targets/x86_64/diffconfig.txt | 26 ++++++++++++
>>>   2 files changed, 102 insertions(+)
>>>   create mode 100755 buildme.sh
>>>   create mode 100644 buildme_targets/x86_64/diffconfig.txt
>>> 
>> 
>> 
>> While I've written similar scripts for helping colleagues
>> bootstrap a build, I'm not entirely convinced that adding more
>> piles of shell script to the build tree is necessarily an
>> improvement, if the goal is helping people build an image that
>> suits them.
>
> My goal is more to have an easy way to build a useful image
> that should work for most people.  Lots of people and organizations
> have their own forks and scripts, and it seems much of the changes are just
> tweaks how to the core project is built.  If we can bring some
> of this into the main tree maybe that will help everyone work
> closer to upstream code.

rather than making a bunch of scripts for specific models, how about some 
scripts that will let you build from a downloaded .config (yes, you can do this 
manually, but it could be greatly simplified)

Then people can build the latest code from a much larger set of configs, and 
tweak a config and then use it over time to build newer versions.

David Lang
Bastian Bittorf May 28, 2016, 12:02 p.m. UTC | #5
* David Lang <david@lang.hm> [28.05.2016 13:48]:
> rather than making a bunch of scripts for specific models, how about
> some scripts that will let you build from a downloaded .config (yes,
> you can do this manually, but it could be greatly simplified)

but for this, you cont need a script at all. its just:

cp $download_config .config
make defconfig
make

is'n it?

bye, bastian
Ben Greear May 28, 2016, 2:52 p.m. UTC | #6
On 05/28/2016 05:02 AM, Bastian Bittorf wrote:
> * David Lang <david@lang.hm> [28.05.2016 13:48]:
>> rather than making a bunch of scripts for specific models, how about
>> some scripts that will let you build from a downloaded .config (yes,
>> you can do this manually, but it could be greatly simplified)
>
> but for this, you cont need a script at all. its just:
>
> cp $download_config .config
> make defconfig
> make
>
> is'n it?

You also need to install feeds as far as I understand.

Having full .configs are likely to rot faster than just diff-configs,
so that is why I like just storing the diffconfigs.  And, having them
local and tracked in git and such just makes everything easier.

With regards to feeds, the feeds to use might can be automatically
determined by the diffconfig file, or maybe just another meta-data
file in the buildme dir.

Thanks,
Ben
Ben Greear May 28, 2016, 2:55 p.m. UTC | #7
On 05/27/2016 11:10 PM, David Lang wrote:
> On Fri, 27 May 2016, Ben Greear wrote:
>
>> On 05/27/2016 02:46 AM, Karl Palsson wrote:
>>>
>>> greearb@candelatech.com wrote:
>>>> From: Ben Greear <greearb@candelatech.com>
>>>>
>>>> The idea is to be able to allow newbies to easily build images
>>>> for common hardware. These images should be user-friendly,
>>>> including luci and other tools that may aid debugging and use
>>>> of the platform.
>>>>
>>>> Signed-off-by: Ben Greear <greearb@candelatech.com>
>>>> ---
>>>>   buildme.sh                            | 76 +++++++++++++++++++++++++++++++++++
>>>>   buildme_targets/x86_64/diffconfig.txt | 26 ++++++++++++
>>>>   2 files changed, 102 insertions(+)
>>>>   create mode 100755 buildme.sh
>>>>   create mode 100644 buildme_targets/x86_64/diffconfig.txt
>>>>
>>>
>>>
>>> While I've written similar scripts for helping colleagues
>>> bootstrap a build, I'm not entirely convinced that adding more
>>> piles of shell script to the build tree is necessarily an
>>> improvement, if the goal is helping people build an image that
>>> suits them.
>>
>> My goal is more to have an easy way to build a useful image
>> that should work for most people.  Lots of people and organizations
>> have their own forks and scripts, and it seems much of the changes are just
>> tweaks how to the core project is built.  If we can bring some
>> of this into the main tree maybe that will help everyone work
>> closer to upstream code.
>
> rather than making a bunch of scripts for specific models, how about some scripts that will let you build from a downloaded .config (yes, you can do this manually, but it could be greatly simplified)
>
> Then people can build the latest code from a much larger set of configs, and tweak a config and then use it over time to build newer versions.

My idea was to store these configs in the git repo and have an easy way for users
to find and use them.

The x86_64 is just a starting point.  I think there could be a large amount of different
configs, for different boards, different needs, etc.  Basically, everyone using openwrt has
a unique config...I think a lot of that could be re-used and shared.

Thanks,
Ben



>
> David Lang
>
David Lang May 28, 2016, 10:22 p.m. UTC | #8
On Sat, 28 May 2016, Bastian Bittorf wrote:
> * David Lang <david@lang.hm> [28.05.2016 13:48]:
>> rather than making a bunch of scripts for specific models, how about
>> some scripts that will let you build from a downloaded .config (yes,
>> you can do this manually, but it could be greatly simplified)
>
> but for this, you cont need a script at all. its just:
>
> cp $download_config .config
> make defconfig
> make
>
> is'n it?

Is it? don't you have to do a make oldconfig if it's been a while since the 
defconfig was created.

none of this is hard for people who are used to doing it, but it could be 
simplified for people who are compiling it for the first time.

Just a script that sets up the correct compile environment, sets up the feeds 
would be a good start.

making things so that you can have a couple different configs and the resulting 
images reasonably.

David Lang
Conor O'Gorman May 31, 2016, 11:42 a.m. UTC | #9
On 28/05/16 15:55, Ben Greear wrote:
>
>
> On 05/27/2016 11:10 PM, David Lang wrote:
>> On Fri, 27 May 2016, Ben Greear wrote:
>>
>>> On 05/27/2016 02:46 AM, Karl Palsson wrote:
>>>>
>>>> greearb@candelatech.com wrote:
>>>>> From: Ben Greear <greearb@candelatech.com>
>>>>>
>>>>> The idea is to be able to allow newbies to easily build images
>>>>> for common hardware. These images should be user-friendly,
>>>>> including luci and other tools that may aid debugging and use
>>>>> of the platform.
>>>>>
>>>>> Signed-off-by: Ben Greear <greearb@candelatech.com>
>>>>> ---
>>>>>   buildme.sh                            | 76 
>>>>> +++++++++++++++++++++++++++++++++++
>>>>>   buildme_targets/x86_64/diffconfig.txt | 26 ++++++++++++
>>>>>   2 files changed, 102 insertions(+)
>>>>>   create mode 100755 buildme.sh
>>>>>   create mode 100644 buildme_targets/x86_64/diffconfig.txt
>>>>>
>>>>
>>>>
>>>> While I've written similar scripts for helping colleagues
>>>> bootstrap a build, I'm not entirely convinced that adding more
>>>> piles of shell script to the build tree is necessarily an
>>>> improvement, if the goal is helping people build an image that
>>>> suits them.
>>>
>>> My goal is more to have an easy way to build a useful image
>>> that should work for most people.  Lots of people and organizations
>>> have their own forks and scripts, and it seems much of the changes 
>>> are just
>>> tweaks how to the core project is built.  If we can bring some
>>> of this into the main tree maybe that will help everyone work
>>> closer to upstream code.
>>
>> rather than making a bunch of scripts for specific models, how about 
>> some scripts that will let you build from a downloaded .config (yes, 
>> you can do this manually, but it could be greatly simplified)
>>
>> Then people can build the latest code from a much larger set of 
>> configs, and tweak a config and then use it over time to build newer 
>> versions.
>
> My idea was to store these configs in the git repo and have an easy 
> way for users
> to find and use them.
>
> The x86_64 is just a starting point.  I think there could be a large 
> amount of different
> configs, for different boards, different needs, etc.  Basically, 
> everyone using openwrt has
> a unique config...I think a lot of that could be re-used and shared.
>
> Thanks,
> Ben
>
>
The ImageBuilder should be covering this scenario.

Conor
John Crispin June 1, 2016, 6:07 p.m. UTC | #10
Hi Ben,

also inclined to reject this one. it will open up the pandoras box and
we will end up maintaining piles of diffconfig files. it would make
morse sense to document what the script does inside web.git as a "how to
build" or "getting started" page

	John

On 27/05/2016 01:57, greearb@candelatech.com wrote:
> From: Ben Greear <greearb@candelatech.com>
> 
> The idea is to be able to allow newbies to easily build images
> for common hardware.  These images should be user-friendly, including
> luci and other tools that may aid debugging and use of the platform.
> 
> Signed-off-by: Ben Greear <greearb@candelatech.com>
> ---
>  buildme.sh                            | 76 +++++++++++++++++++++++++++++++++++
>  buildme_targets/x86_64/diffconfig.txt | 26 ++++++++++++
>  2 files changed, 102 insertions(+)
>  create mode 100755 buildme.sh
>  create mode 100644 buildme_targets/x86_64/diffconfig.txt
> 
> diff --git a/buildme.sh b/buildme.sh
> new file mode 100755
> index 0000000..83644fa
> --- /dev/null
> +++ b/buildme.sh
> @@ -0,0 +1,76 @@
> +#!/usr/bin/env bash
> +
> +USAGE=$( cat <<EOF
> +Usage:\n
> +# -t:  Target\n
> +# -l:  List available targets.\n
> +# -f:  Skip feeds (useful on rebuild)\n
> +# -j:  Compile jobs (default is 8)\n
> +# -h:  Show help and exit\n
> +#\n
> +#\n
> +# Example:  Build an image for x86_64
> +#  ./buildme.sh -t x86_64\n
> +EOF
> +)
> +
> +BMT=buildme_targets
> +TARGET=UNKNOWN
> +LIST_TARGETS=0
> +SKIP_FEEDS=0
> +JOBS=8
> +
> +while getopts "t:j:lfh" flag
> +  do
> +  case $flag in
> +      t) TARGET=$OPTARG;;
> +      l) LIST_TARGETS=1;;
> +      f) SKIP_FEEDS=1;;
> +      j) JOBS=$OPTARG;;
> +      h) echo -e USAGE && exit 0;;
> +
> +      *) echo "Un-known option: $flag"; echo -e $USAGE;exit 1;;
> +  esac
> +done
> +
> +if [ $LIST_TARGETS == 1 ]
> +then
> +    ls $BMT
> +    exit 0
> +fi
> +
> +echo "Target dir: $BMT/$TARGET"
> +
> +if [ ! -d $BMT/$TARGET ]
> +then
> +    echo "ERROR:  Unknown target, try -l option?"
> +    exit 1;
> +fi
> +
> +set -x
> +
> +if [ $SKIP_FEEDS != 1 ]
> +then
> +    # Update feeds
> +    ./scripts/feeds update -a || exit 1
> +
> +    # Enable luci web interface
> +    ./scripts/feeds install -d y luci || exit 1
> +
> +    # Enable ethtool for driver info and stats and such
> +    ./scripts/feeds install -d y ethtool || exit 1
> +fi
> +
> +# Copy default config into place
> +cp $BMT/$TARGET/diffconfig.txt .config || exit 1
> +
> +# Build complate .config file based on whatever is latest, etc.
> +make defconfig || exit 1
> +
> +# Build
> +make v=S -j $JOBS || exit 1
> +
> +# Try to figure out where the images are at.
> +set +x
> +. .config > /dev/null 2>&1
> +echo "Images may be found at: bin/targets/$CONFIG_TARGET_BOARD/$CONFIG_TARGET_SUBTARGET/"
> diff --git a/buildme_targets/x86_64/diffconfig.txt b/buildme_targets/x86_64/diffconfig.txt
> new file mode 100644
> index 0000000..d3db402
> --- /dev/null
> +++ b/buildme_targets/x86_64/diffconfig.txt
> @@ -0,0 +1,26 @@
> +CONFIG_TARGET_x86=y
> +CONFIG_TARGET_x86_64=y
> +CONFIG_TARGET_x86_64_Generic=y
> +CONFIG_ALL_NONSHARED=y
> +CONFIG_BUSYBOX_CUSTOM=y
> +CONFIG_BUSYBOX_CONFIG_ARP=y
> +CONFIG_BUSYBOX_CONFIG_FEATURE_TELNET_AUTOLOGIN=y
> +CONFIG_BUSYBOX_CONFIG_FEATURE_TELNET_TTYPE=y
> +CONFIG_BUSYBOX_CONFIG_LSPCI=y
> +CONFIG_BUSYBOX_CONFIG_LSUSB=y
> +CONFIG_BUSYBOX_CONFIG_MORE=y
> +CONFIG_BUSYBOX_CONFIG_TELNET=y
> +CONFIG_IMAGEOPT=y
> +CONFIG_PACKAGE_hostapd-common-old=y
> +CONFIG_PACKAGE_hostapd-utils=y
> +CONFIG_PACKAGE_kmod-hwmon-core=y
> +CONFIG_PACKAGE_kmod-i2c-algo-bit=y
> +CONFIG_PACKAGE_kmod-i2c-core=y
> +CONFIG_PACKAGE_kmod-igb=y
> +CONFIG_PACKAGE_kmod-libphy=y
> +CONFIG_PACKAGE_kmod-skge=y
> +CONFIG_PACKAGE_kmod-sky2=y
> +CONFIG_PACKAGE_kmod-tg3=y
> +# CONFIG_PER_FEED_REPO is not set
> +CONFIG_TARGET_KERNEL_PARTSIZE=64
> +CONFIG_TARGET_ROOTFS_PARTSIZE=1024
>
Jo-Philipp Wich June 1, 2016, 6:10 p.m. UTC | #11
Hi,

maybe we can add a well annotated variant of this script into web.git as
part of the documentation tree and do some kind of build script example
page?

~ Jo´
Ben Greear June 1, 2016, 6:21 p.m. UTC | #12
On 06/01/2016 11:07 AM, John Crispin wrote:
> Hi Ben,
>
> also inclined to reject this one. it will open up the pandoras box and
> we will end up maintaining piles of diffconfig files. it would make
> morse sense to document what the script does inside web.git as a "how to
> build" or "getting started" page

I do not have much history with LEDE/WRT, but is the diffconfig output
really that fragile?

I will be willing to maintain the x86-64 and probably Ventana
diffconfigs, and even if it totally fails, you have not lost
any functionality since the main infrastructure is unchanged.

I think I could further improve the buildme.sh to automatically
pull in the feeds based on the packages in the diffconfig too,
which should further automate building and make the buildme.sh
more generic for different platforms.

For jow's idea, putting it in a web page may be better than nothing,
but then it just means anyone wanting to use this type of thing
has more work to do, and for newbies, that can be all the difference
between success and failure.

I still think you should consider letting this patch in.  I can
re-do it based on removing the 1/2 patch, so the diffconfig gets
bigger, but we can still build usable apu2 images out of the box.

Either way, thanks for considering the patch.

Thanks,
Ben
John Crispin June 3, 2016, 9:17 a.m. UTC | #13
On 01/06/2016 20:21, Ben Greear wrote:
> On 06/01/2016 11:07 AM, John Crispin wrote:
>> Hi Ben,
>>
>> also inclined to reject this one. it will open up the pandoras box and
>> we will end up maintaining piles of diffconfig files. it would make
>> morse sense to document what the script does inside web.git as a "how to
>> build" or "getting started" page
> 
> I do not have much history with LEDE/WRT, but is the diffconfig output
> really that fragile?
> 
> I will be willing to maintain the x86-64 and probably Ventana
> diffconfigs, and even if it totally fails, you have not lost
> any functionality since the main infrastructure is unchanged.

for how long and will all the other people submitting these files also
provide LTS ? normally activism fades after a few months. we can see
this will board support and package maintenance. i think this will be a
burden.

adding a "how to build" page to web.git is a good idea and we can
consider to put all the diffconfig files some place. but i think that
source.git is not the right place.

	John


> 
> I think I could further improve the buildme.sh to automatically
> pull in the feeds based on the packages in the diffconfig too,
> which should further automate building and make the buildme.sh
> more generic for different platforms.
> 
> For jow's idea, putting it in a web page may be better than nothing,
> but then it just means anyone wanting to use this type of thing
> has more work to do, and for newbies, that can be all the difference
> between success and failure.
> 
> I still think you should consider letting this patch in.  I can
> re-do it based on removing the 1/2 patch, so the diffconfig gets
> bigger, but we can still build usable apu2 images out of the box.
> 
> Either way, thanks for considering the patch.
> 
> Thanks,
> Ben
>
Ben Greear June 3, 2016, 12:27 p.m. UTC | #14
On 06/03/2016 02:17 AM, John Crispin wrote:
>
>
> On 01/06/2016 20:21, Ben Greear wrote:
>> On 06/01/2016 11:07 AM, John Crispin wrote:
>>> Hi Ben,
>>>
>>> also inclined to reject this one. it will open up the pandoras box and
>>> we will end up maintaining piles of diffconfig files. it would make
>>> morse sense to document what the script does inside web.git as a "how to
>>> build" or "getting started" page
>>
>> I do not have much history with LEDE/WRT, but is the diffconfig output
>> really that fragile?
>>
>> I will be willing to maintain the x86-64 and probably Ventana
>> diffconfigs, and even if it totally fails, you have not lost
>> any functionality since the main infrastructure is unchanged.
>
> for how long and will all the other people submitting these files also
> provide LTS ? normally activism fades after a few months. we can see
> this will board support and package maintenance. i think this will be a
> burden.

Anything that makes it more easy for others to build images
should help bring in more newbies faster, which might keep the
activism going, especially for common platforms that will
constantly be getting new hardware (like x86-64).

Currently, a newbie with an x86-64 board is going to have a slow
and frustrating time trying to get any sort of useful 'AP' built
from LEDE.  I think I can help to fix that problem if you will
let me.

So, how about you accept a small number of diffconfig recipes, and the buildme
script, and then see how that works for a while.  I can update
buildme.sh to grab a diffconfig from a URL (using curl, or something)
and then folks wanting to provide out-of-tree diffconfigs need only
put their config in a public place and tell users how to use it:

./buildme.sh -T http://my.funkyplatform.com/board1

Thanks,
Ben
Conor O'Gorman June 3, 2016, 12:37 p.m. UTC | #15
On 03/06/16 13:27, Ben Greear wrote:
>
>
> On 06/03/2016 02:17 AM, John Crispin wrote:
>>
>>
>> On 01/06/2016 20:21, Ben Greear wrote:
>>> On 06/01/2016 11:07 AM, John Crispin wrote:
>>>> Hi Ben,
>>>>
>>>> also inclined to reject this one. it will open up the pandoras box and
>>>> we will end up maintaining piles of diffconfig files. it would make
>>>> morse sense to document what the script does inside web.git as a 
>>>> "how to
>>>> build" or "getting started" page
>>>
>>> I do not have much history with LEDE/WRT, but is the diffconfig output
>>> really that fragile?
>>>
>>> I will be willing to maintain the x86-64 and probably Ventana
>>> diffconfigs, and even if it totally fails, you have not lost
>>> any functionality since the main infrastructure is unchanged.
>>
>> for how long and will all the other people submitting these files also
>> provide LTS ? normally activism fades after a few months. we can see
>> this will board support and package maintenance. i think this will be a
>> burden.
>
> Anything that makes it more easy for others to build images
> should help bring in more newbies faster, which might keep the
> activism going, especially for common platforms that will
> constantly be getting new hardware (like x86-64).
>
> Currently, a newbie with an x86-64 board is going to have a slow
> and frustrating time trying to get any sort of useful 'AP' built
> from LEDE.  I think I can help to fix that problem if you will
> let me.
>
> So, how about you accept a small number of diffconfig recipes, and the 
> buildme
> script, and then see how that works for a while.  I can update
> buildme.sh to grab a diffconfig from a URL (using curl, or something)
> and then folks wanting to provide out-of-tree diffconfigs need only
> put their config in a public place and tell users how to use it:
>
> ./buildme.sh -T http://my.funkyplatform.com/board1
>
> Thanks,
> Ben
>
This seems to duplicate the functionality of the sub-targets and 
profiles mechanism?
Ben Greear June 3, 2016, 12:54 p.m. UTC | #16
On 06/03/2016 05:37 AM, Conor O'Gorman wrote:
>
>
> On 03/06/16 13:27, Ben Greear wrote:
>>
>>
>> On 06/03/2016 02:17 AM, John Crispin wrote:
>>>
>>>
>>> On 01/06/2016 20:21, Ben Greear wrote:
>>>> On 06/01/2016 11:07 AM, John Crispin wrote:
>>>>> Hi Ben,
>>>>>
>>>>> also inclined to reject this one. it will open up the pandoras box and
>>>>> we will end up maintaining piles of diffconfig files. it would make
>>>>> morse sense to document what the script does inside web.git as a "how to
>>>>> build" or "getting started" page
>>>>
>>>> I do not have much history with LEDE/WRT, but is the diffconfig output
>>>> really that fragile?
>>>>
>>>> I will be willing to maintain the x86-64 and probably Ventana
>>>> diffconfigs, and even if it totally fails, you have not lost
>>>> any functionality since the main infrastructure is unchanged.
>>>
>>> for how long and will all the other people submitting these files also
>>> provide LTS ? normally activism fades after a few months. we can see
>>> this will board support and package maintenance. i think this will be a
>>> burden.
>>
>> Anything that makes it more easy for others to build images
>> should help bring in more newbies faster, which might keep the
>> activism going, especially for common platforms that will
>> constantly be getting new hardware (like x86-64).
>>
>> Currently, a newbie with an x86-64 board is going to have a slow
>> and frustrating time trying to get any sort of useful 'AP' built
>> from LEDE.  I think I can help to fix that problem if you will
>> let me.
>>
>> So, how about you accept a small number of diffconfig recipes, and the buildme
>> script, and then see how that works for a while.  I can update
>> buildme.sh to grab a diffconfig from a URL (using curl, or something)
>> and then folks wanting to provide out-of-tree diffconfigs need only
>> put their config in a public place and tell users how to use it:
>>
>> ./buildme.sh -T http://my.funkyplatform.com/board1
>>
>> Thanks,
>> Ben
>>
> This seems to duplicate the functionality of the sub-targets and profiles mechanism?

Maybe some of it, but for more generic platforms, I am not sure sub-targets make
sense, and in general, it is more difficult to edit those than to change a diffconfig
as far as I can tell.

Thanks,
Ben
Karl Palsson June 3, 2016, 4:16 p.m. UTC | #17
Conor O'Gorman <i@conorogorman.net> wrote:
> 
> 
> On 03/06/16 13:27, Ben Greear wrote:
> >
> >
> > On 06/03/2016 02:17 AM, John Crispin wrote:
> >>
> >>
> >> On 01/06/2016 20:21, Ben Greear wrote:
> >>> On 06/01/2016 11:07 AM, John Crispin wrote:
> >>>> Hi Ben,
> >>>>
> >>>> also inclined to reject this one. it will open up the pandoras box and
> >>>> we will end up maintaining piles of diffconfig files. it would make
> >>>> morse sense to document what the script does inside web.git as a 
> >>>> "how to
> >>>> build" or "getting started" page
> >>>
> >>> I do not have much history with LEDE/WRT, but is the diffconfig output
> >>> really that fragile?
> >>>
> >>> I will be willing to maintain the x86-64 and probably Ventana
> >>> diffconfigs, and even if it totally fails, you have not lost
> >>> any functionality since the main infrastructure is unchanged.
> >>
> >> for how long and will all the other people submitting these files also
> >> provide LTS ? normally activism fades after a few months. we can see
> >> this will board support and package maintenance. i think this will be a
> >> burden.
> >
> > Anything that makes it more easy for others to build images
> > should help bring in more newbies faster, which might keep the
> > activism going, especially for common platforms that will
> > constantly be getting new hardware (like x86-64).
> >
> > Currently, a newbie with an x86-64 board is going to have a slow
> > and frustrating time trying to get any sort of useful 'AP' built
> > from LEDE.  I think I can help to fix that problem if you will
> > let me.
> >
> > So, how about you accept a small number of diffconfig recipes, and the 
> > buildme
> > script, and then see how that works for a while.  I can update
> > buildme.sh to grab a diffconfig from a URL (using curl, or something)
> > and then folks wanting to provide out-of-tree diffconfigs need only
> > put their config in a public place and tell users how to use it:
> >
> > ./buildme.sh -T http://my.funkyplatform.com/board1
> >
> > Thanks,
> > Ben
> >
> This seems to duplicate the functionality of the sub-targets
> and profiles mechanism?


Also, image builder.
Karl Palsson June 3, 2016, 4:19 p.m. UTC | #18
> Maybe some of it, but for more generic platforms, I am not sure
> sub-targets make sense, and in general, it is more difficult to
> edit those than to change a diffconfig as far as I can tell.

For who?  I'm pretty sure I can use "make menuconfig" _far_ more reliably than I can edit a diffconfig file to use a "buildme" script.  If I don't know anything, and _need_ to build a custom image that can't be built with image builder, or post install config, then menuconfig at least makes sure that dependencies get pulled in properly.  Adding line items to file that you then "defoncfig" and hope you got perfectt seems fraught with danger, and for what? To avoid showing someone how to use menuconfig?  Which they'll need to use anyway?  Because I don't believe for a _second_ that somehow you're going to have "buildme" magic that automatically solves all the configs someone needs.  

Sincerely,
Karl Palsson
Ben Greear June 3, 2016, 4:32 p.m. UTC | #19
On 06/03/2016 09:19 AM, Karl Palsson wrote:
>
>> Maybe some of it, but for more generic platforms, I am not sure
>> sub-targets make sense, and in general, it is more difficult to
>> edit those than to change a diffconfig as far as I can tell.
>
> For who?  I'm pretty sure I can use "make menuconfig" _far_ more reliably than I can edit a diffconfig file to use a "buildme" script.  If I don't know anything, and _need_ to build a custom image that can't be built with image builder, or post install config, then menuconfig at least makes sure that dependencies get pulled in properly.  Adding line items to file that you then "defoncfig" and hope you got perfectt seems fraught with danger, and for what? To avoid showing someone how to use menuconfig?  Which they'll need to use anyway?  Because I don't believe for a _second_ that somehow you're going to have "buildme" magic that automatically solves all the configs someone needs.

First of all, any time you try to add a new module, someone will bitch about
how it bloats their beautiful image and makes it impure.  Adding more sub-targets
almost cannot by definition be any less likely to rot or be un-maintained than
the buildme targets I am proposing.

To create (or update) the diffconfig, you can run menuconfig, twiddle as you like, and then save the
new diffconfig.  And commit changes (assuming you are maintaining this buildme profile).

If the diffconfig profiles are really that un-cool, then maybe just add the buildme
script with ability to use URL for the diffconfig profile and let folks host them
elsewhere.  That is less user-friendly (it would be hard to show user all available options),
but at least it is still one command to build an image.

And anyway, no one is proposing to remove menuconfig logic, I just want to add some
wrappers to make it easier for newbies that may initially be bewildered by all of the menuconfig
options (and lack of options if you haven't installed the proper feeds already).

Thanks,
Ben
diff mbox

Patch

diff --git a/buildme.sh b/buildme.sh
new file mode 100755
index 0000000..83644fa
--- /dev/null
+++ b/buildme.sh
@@ -0,0 +1,76 @@ 
+#!/usr/bin/env bash
+
+USAGE=$( cat <<EOF
+Usage:\n
+# -t:  Target\n
+# -l:  List available targets.\n
+# -f:  Skip feeds (useful on rebuild)\n
+# -j:  Compile jobs (default is 8)\n
+# -h:  Show help and exit\n
+#\n
+#\n
+# Example:  Build an image for x86_64
+#  ./buildme.sh -t x86_64\n
+EOF
+)
+
+BMT=buildme_targets
+TARGET=UNKNOWN
+LIST_TARGETS=0
+SKIP_FEEDS=0
+JOBS=8
+
+while getopts "t:j:lfh" flag
+  do
+  case $flag in
+      t) TARGET=$OPTARG;;
+      l) LIST_TARGETS=1;;
+      f) SKIP_FEEDS=1;;
+      j) JOBS=$OPTARG;;
+      h) echo -e USAGE && exit 0;;
+
+      *) echo "Un-known option: $flag"; echo -e $USAGE;exit 1;;
+  esac
+done
+
+if [ $LIST_TARGETS == 1 ]
+then
+    ls $BMT
+    exit 0
+fi
+
+echo "Target dir: $BMT/$TARGET"
+
+if [ ! -d $BMT/$TARGET ]
+then
+    echo "ERROR:  Unknown target, try -l option?"
+    exit 1;
+fi
+
+set -x
+
+if [ $SKIP_FEEDS != 1 ]
+then
+    # Update feeds
+    ./scripts/feeds update -a || exit 1
+
+    # Enable luci web interface
+    ./scripts/feeds install -d y luci || exit 1
+
+    # Enable ethtool for driver info and stats and such
+    ./scripts/feeds install -d y ethtool || exit 1
+fi
+
+# Copy default config into place
+cp $BMT/$TARGET/diffconfig.txt .config || exit 1
+
+# Build complate .config file based on whatever is latest, etc.
+make defconfig || exit 1
+
+# Build
+make v=S -j $JOBS || exit 1
+
+# Try to figure out where the images are at.
+set +x
+. .config > /dev/null 2>&1
+echo "Images may be found at: bin/targets/$CONFIG_TARGET_BOARD/$CONFIG_TARGET_SUBTARGET/"
diff --git a/buildme_targets/x86_64/diffconfig.txt b/buildme_targets/x86_64/diffconfig.txt
new file mode 100644
index 0000000..d3db402
--- /dev/null
+++ b/buildme_targets/x86_64/diffconfig.txt
@@ -0,0 +1,26 @@ 
+CONFIG_TARGET_x86=y
+CONFIG_TARGET_x86_64=y
+CONFIG_TARGET_x86_64_Generic=y
+CONFIG_ALL_NONSHARED=y
+CONFIG_BUSYBOX_CUSTOM=y
+CONFIG_BUSYBOX_CONFIG_ARP=y
+CONFIG_BUSYBOX_CONFIG_FEATURE_TELNET_AUTOLOGIN=y
+CONFIG_BUSYBOX_CONFIG_FEATURE_TELNET_TTYPE=y
+CONFIG_BUSYBOX_CONFIG_LSPCI=y
+CONFIG_BUSYBOX_CONFIG_LSUSB=y
+CONFIG_BUSYBOX_CONFIG_MORE=y
+CONFIG_BUSYBOX_CONFIG_TELNET=y
+CONFIG_IMAGEOPT=y
+CONFIG_PACKAGE_hostapd-common-old=y
+CONFIG_PACKAGE_hostapd-utils=y
+CONFIG_PACKAGE_kmod-hwmon-core=y
+CONFIG_PACKAGE_kmod-i2c-algo-bit=y
+CONFIG_PACKAGE_kmod-i2c-core=y
+CONFIG_PACKAGE_kmod-igb=y
+CONFIG_PACKAGE_kmod-libphy=y
+CONFIG_PACKAGE_kmod-skge=y
+CONFIG_PACKAGE_kmod-sky2=y
+CONFIG_PACKAGE_kmod-tg3=y
+# CONFIG_PER_FEED_REPO is not set
+CONFIG_TARGET_KERNEL_PARTSIZE=64
+CONFIG_TARGET_ROOTFS_PARTSIZE=1024