From patchwork Sun Dec 16 16:56:24 2012 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: "Yann E. MORIN" X-Patchwork-Id: 206701 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from silver.osuosl.org (silver.osuosl.org [140.211.166.136]) by ozlabs.org (Postfix) with ESMTP id 3F8142C007B for ; Mon, 17 Dec 2012 03:59:16 +1100 (EST) Received: from localhost (localhost [127.0.0.1]) by silver.osuosl.org (Postfix) with ESMTP id B5F9A26376; Sun, 16 Dec 2012 16:59:14 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org X-Amavis-Alert: BAD HEADER SECTION, Duplicate header field: "References" Received: from silver.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id zrm3cd9HZPKi; Sun, 16 Dec 2012 16:59:03 +0000 (UTC) Received: from ash.osuosl.org (ash.osuosl.org [140.211.166.34]) by silver.osuosl.org (Postfix) with ESMTP id EA3092064D; Sun, 16 Dec 2012 16:58:14 +0000 (UTC) X-Original-To: buildroot@lists.busybox.net Delivered-To: buildroot@osuosl.org Received: from whitealder.osuosl.org (whitealder.osuosl.org [140.211.166.138]) by ash.osuosl.org (Postfix) with ESMTP id 66D998F74B for ; Sun, 16 Dec 2012 16:57:54 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by whitealder.osuosl.org (Postfix) with ESMTP id D1F858AC5E for ; Sun, 16 Dec 2012 16:57:48 +0000 (UTC) X-Virus-Scanned: amavisd-new at osuosl.org X-Amavis-Alert: BAD HEADER SECTION, Duplicate header field: "References" Received: from whitealder.osuosl.org ([127.0.0.1]) by localhost (.osuosl.org [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id aMSUJCg+Bdvk for ; Sun, 16 Dec 2012 16:57:32 +0000 (UTC) X-Greylist: domain auto-whitelisted by SQLgrey-1.7.6 Received: from mail-wi0-f171.google.com (mail-wi0-f171.google.com [209.85.212.171]) by whitealder.osuosl.org (Postfix) with ESMTPS id 49DC18AD1E for ; Sun, 16 Dec 2012 16:57:06 +0000 (UTC) Received: by mail-wi0-f171.google.com with SMTP id hn14so1456996wib.10 for ; Sun, 16 Dec 2012 08:57:04 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=sender:from:to:cc:subject:date:message-id:x-mailer:in-reply-to :references:in-reply-to:references; bh=v5LPgLwOPqwCDjdFEkhMlqN3MAxhnDi7n1bcyuChnOc=; b=cSUTA/Cu81YSXucEkn+3oTrs2a2r0ppeWewT5HFi7/cspOIVenOsEFss7lzM3KwXMP P6hW6JzIP3NTow9ZoM/Lna3QDNe2R6bj7hyJtqs1Qt/yLzY5/DcYuGO0DPwj33PCC1WU Fh3Lhu5+oEsx15uxyGhuEOjMZBjII4Mba/UB4EEgHTgg3J+9HfkGwhAT947p98ZK14rR HuPSWETcmJpxWYgKx2vxqmPMkpqpLyqwj4Apo4zpCdK5dTOFd7lo9vPzsXvpwdfIrWLy kBeISkLzMlsxY6HGeOfXIZUsplHACPHE5fldRb3kkOV2ndalpGTBPVjmfwa+HTJFxRiu cVSg== Received: by 10.180.107.72 with SMTP id ha8mr11555552wib.21.1355677024677; Sun, 16 Dec 2012 08:57:04 -0800 (PST) Received: from localhost.localdomain (ARennes-256-1-41-119.w90-32.abo.wanadoo.fr. [90.32.24.119]) by mx.google.com with ESMTPS id bz12sm8243078wib.5.2012.12.16.08.57.02 (version=TLSv1/SSLv3 cipher=OTHER); Sun, 16 Dec 2012 08:57:03 -0800 (PST) From: "Yann E. MORIN" To: buildroot@busybox.net Date: Sun, 16 Dec 2012 17:56:24 +0100 Message-Id: X-Mailer: git-send-email 1.7.2.5 In-Reply-To: References: In-Reply-To: References: Cc: "Yann E. MORIN" Subject: [Buildroot] [PATCH 17/19] package/ceph: new package X-BeenThere: buildroot@busybox.net X-Mailman-Version: 2.1.14 Precedence: list List-Id: Discussion and development of buildroot List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Errors-To: buildroot-bounces@busybox.net Sender: buildroot-bounces@busybox.net Signed-off-by: "Yann E. MORIN" Cc: Peter Korsgaard --- package/Config.in | 1 + package/ceph/Config.in | 21 +++++++++ package/ceph/ceph-no-envz.patch | 63 ++++++++++++++++++++++++++++ package/ceph/ceph-no-getloadavg.patch | 61 +++++++++++++++++++++++++++ package/ceph/ceph-no-posix_fallocate.patch | 50 ++++++++++++++++++++++ package/ceph/ceph.mk | 53 +++++++++++++++++++++++ 6 files changed, 249 insertions(+), 0 deletions(-) create mode 100644 package/ceph/Config.in create mode 100644 package/ceph/ceph-no-envz.patch create mode 100644 package/ceph/ceph-no-getloadavg.patch create mode 100644 package/ceph/ceph-no-posix_fallocate.patch create mode 100644 package/ceph/ceph.mk diff --git a/package/Config.in b/package/Config.in index 04aef11..657f568 100644 --- a/package/Config.in +++ b/package/Config.in @@ -590,6 +590,7 @@ source "package/bind/Config.in" source "package/bmon/Config.in" source "package/bridge-utils/Config.in" source "package/can-utils/Config.in" +source "package/ceph/Config.in" source "package/connman/Config.in" source "package/ctorrent/Config.in" source "package/conntrack-tools/Config.in" diff --git a/package/ceph/Config.in b/package/ceph/Config.in new file mode 100644 index 0000000..61f1947 --- /dev/null +++ b/package/ceph/Config.in @@ -0,0 +1,21 @@ +comment "ceph requires C++, largefiles, threads, wide chars" + depends on !BR2_INSTALL_LIBSTDCPP || !BR2_LARGEFILE || !BR2_TOOLCHAIN_HAS_THREADS || !BR2_USE_WCHAR + +config BR2_PACKAGE_CEPH + bool "ceph" + depends on BR2_INSTALL_LIBSTDCPP # boost + depends on BR2_LARGEFILE # boost, libnss, util-linux + depends on BR2_TOOLCHAIN_HAS_THREADS # libnss + depends on BR2_USE_WCHAR # util-linux + select BR2_PACKAGE_BOOST + select BR2_PACKAGE_EXPAT + select BR2_PACKAGE_KEYUTILS + select BR2_PACKAGE_LIBEDIT2 + select BR2_PACKAGE_LIBNSS + select BR2_PACKAGE_UTIL_LINUX + select BR2_PACKAGE_UTIL_LINUX_LIBUUID + help + Ceph is a distributed object store and file system designed to + provide excellent performance, reliability and scalability. + + http://ceph.com/ diff --git a/package/ceph/ceph-no-envz.patch b/package/ceph/ceph-no-envz.patch new file mode 100644 index 0000000..c0369ba --- /dev/null +++ b/package/ceph/ceph-no-envz.patch @@ -0,0 +1,63 @@ +tools+base: do not include envz.h on uClibc + +uClibc does not have envz, and no envz function is in fact used. +So, simply do not include it under uClibc. + +Signed-off-by: "Yann E. MORIN" + +--- ceph-0.51.orig/src/tools/common.cc 2012-09-09 00:13:32.128838177 +0200 ++++ ceph-0.51/src/tools/common.cc 2012-09-09 00:13:59.432064556 +0200 +@@ -17,9 +17,9 @@ + #include + using namespace std; + +-#if !defined(DARWIN) && !defined(__FreeBSD__) ++#if !defined(DARWIN) && !defined(__FreeBSD__) && !defined(__UCLIBC__) + #include +-#endif // DARWIN ++#endif // DARWIN || __UCLIBC__ + + #include + #include +--- ceph-0.51.orig/src/ceph_syn.cc 2012-09-09 00:25:23.302722032 +0200 ++++ ceph-0.51/src/ceph_syn.cc 2012-09-09 00:24:39.702892463 +0200 +@@ -31,9 +31,9 @@ + #include "common/ceph_argparse.h" + #include "common/pick_address.h" + +-#if !defined(DARWIN) && !defined(__FreeBSD__) ++#if !defined(DARWIN) && !defined(__FreeBSD__) && !defined(__UCLIBC__) + #include +-#endif // DARWIN || __FreeBSD__ ++#endif // DARWIN || __FreeBSD__ || __UCLIBC__ + + #include + #include +--- ceph-0.51.orig/src/ceph_fuse.cc 2012-09-09 00:43:46.875065556 +0200 ++++ ceph-0.51/src/ceph_fuse.cc 2012-09-09 00:44:21.161597834 +0200 +@@ -32,9 +32,9 @@ + #include "global/global_init.h" + #include "common/safe_io.h" + +-#ifndef DARWIN ++#if !defined(DARWIN) && !defined(__UCLIBC__) + #include +-#endif // DARWIN ++#endif // DARWIN || __UCLIBC__ + + #include + #include +--- ceph-0.51.orig/src/testmsgr.cc 2012-09-09 00:44:48.108158886 +0200 ++++ ceph-0.51/src/testmsgr.cc 2012-09-09 00:45:18.218040882 +0200 +@@ -28,9 +28,9 @@ + #include "global/global_init.h" + #include "common/ceph_argparse.h" + +-#ifndef DARWIN ++#if !defined(DARWIN) && !defined(__UCLIBC__) + #include +-#endif // DARWIN ++#endif // DARWIN || __UCLIBC__ + + #include + #include diff --git a/package/ceph/ceph-no-getloadavg.patch b/package/ceph/ceph-no-getloadavg.patch new file mode 100644 index 0000000..7e887df --- /dev/null +++ b/package/ceph/ceph-no-getloadavg.patch @@ -0,0 +1,61 @@ +OSD: do not use getloadavg(3) on uClibc + +getloadavg(3) is not available on uClibc, so do not use it. + +getloadavg is used in two places: + - once to log the loadavg in the heartbeat logs: we simply remove this; + - once to decide whether to schedule scrub (whatever it is) or not: we + simply use the existing coin-flip random choice in this case. + +The second change will probably have an impact on performances... :-( + +Signed-off-by: "Yann E. MORIN" + +diff -durN ceph-0.51.orig/src/osd/OSD.cc ceph-0.51/src/osd/OSD.cc +--- ceph-0.51.orig/src/osd/OSD.cc 2012-08-26 00:58:06.000000000 +0200 ++++ ceph-0.51/src/osd/OSD.cc 2012-09-08 23:56:31.396177885 +0200 +@@ -15,6 +15,7 @@ + #include + #include + #include ++#include /* For __ULIBC__ */ + #include + #include + #include +@@ -1942,10 +1943,12 @@ + { + dout(30) << "heartbeat" << dendl; + ++#ifndef __UCLIBC__ + // get CPU load avg + double loadavgs[1]; + if (getloadavg(loadavgs, 1) == 1) + logger->fset(l_osd_loadavg, loadavgs[0]); ++#endif + + dout(30) << "heartbeat checking stats" << dendl; + +@@ -3216,6 +3219,7 @@ + + bool OSDService::scrub_should_schedule() + { ++#ifndef __UCLIBC__ + double loadavgs[1]; + + // TODOSAM: is_active should be conveyed to OSDService +@@ -3249,6 +3253,15 @@ + << " < max " << g_conf->osd_scrub_load_threshold + << " = yes" << dendl; + return loadavgs[0] < g_conf->osd_scrub_load_threshold; ++#else // ! defined __UCLIBC__ ++ bool coin_flip = (rand() % 3) == whoami % 3; ++ if (coin_flip) { ++ dout(20) << "scrub_should_schedule randomly yes" << dendl; ++ } else { ++ dout(20) << "scrub_should_schedule randomly backing off" << dendl; ++ } ++ return coin_flip; ++#endif // ! defined __UCLIBC__ + } + + void OSD::sched_scrub() diff --git a/package/ceph/ceph-no-posix_fallocate.patch b/package/ceph/ceph-no-posix_fallocate.patch new file mode 100644 index 0000000..479705d --- /dev/null +++ b/package/ceph/ceph-no-posix_fallocate.patch @@ -0,0 +1,50 @@ +Do not call posix_fallocate() on uClibc + +Based on a patch by Thomas, for lttng-babeltrace, in which he states: + + uClibc does not implement posix_fallocate(), and posix_fallocate() is + mostly only an hint to the kernel that we will need such or such + amount of space inside a file. So we just don't call posix_fallocate() + when building against uClibc. + +Just do the same here. + +NOTE! IMHO, this patch is not correct, because overcommit (eg. for sparse +files) on the underlying filesystem may cause allocation failures later on, +which is exactly the situation that posix_fallocate is supposed to avoid. +But, as uClibc still lacks posix_fallocate, we have no choice. + +If you really are concerned about this issue, there exists a posix_fallocate +patch that is not upstream, but is used by the AlpineLinux distribution: + http://git.alpinelinux.org/cgit/aports/tree/main/libc0.9.32/posix_fallocate.patch?id=98a000b8e92b3aa1e0b7a5f94a74910bb0993c4d + http://git.alpinelinux.org/cgit/aports/tree/main/libc0.9.32/posix_fallocate-fix.patch?id=98a000b8e92b3aa1e0b7a5f94a74910bb0993c4d + +Signed-off-by: "Yann E. MORIN" + +diff -durN ceph-0.50.orig/src/os/FileJournal.cc ceph-0.50/src/os/FileJournal.cc +--- ceph-0.50.orig/src/os/FileJournal.cc 2012-08-13 18:43:14.000000000 +0200 ++++ ceph-0.50/src/os/FileJournal.cc 2012-09-08 23:07:12.597910027 +0200 +@@ -23,6 +23,7 @@ + #include "include/compat.h" + + #include ++#include + #include + #include + #include +@@ -303,6 +304,7 @@ + << newsize << " bytes: " << cpp_strerror(err) << dendl; + return -err; + } ++#ifndef __UCLIBC__ + ret = ::posix_fallocate(fd, 0, newsize); + if (ret < 0) { + int err = errno; +@@ -310,6 +312,7 @@ + << newsize << " bytes: " << cpp_strerror(err) << dendl; + return -err; + } ++#endif + max_size = newsize; + } + else { diff --git a/package/ceph/ceph.mk b/package/ceph/ceph.mk new file mode 100644 index 0000000..0849837 --- /dev/null +++ b/package/ceph/ceph.mk @@ -0,0 +1,53 @@ +############################################################# +# +# ceph +# +############################################################# + +CEPH_VERSION = 0.51 +CEPH_SOURCE = ceph-$(CEPH_VERSION).tar.bz2 +CEPH_SITE = http://ceph.com/download/ +CEPH_LICENSE = LGPLv2.1 CC-BY-SA Apache-2.0 GPLv2 LGPLv2+ BSD-3c MIT PD +CEPH_LICENSE_FILES = COPYING +CEPH_INSTALL_STAGING = YES + +CEPH_DEPENDENCIES = \ + boost \ + expat \ + keyutils \ + libedit2 \ + libnss \ + util-linux \ + +# We disable everything for now, because the dependency tree can become +# quite deep if we try to enable some features, and I have not tested that. +# We need at least one crypto lib, and the currently only one available in +# BR, that ceph can use, is libnss +CEPH_CONF_OPT = \ + --with-nss \ + --without-profiler \ + --without-debug \ + --without-tcmalloc \ + --without-hadoop \ + --without-libatomic-ops \ + --without-system-leveldb \ + --without-system-libs3 \ + --without-rest-bench \ + --without-cryptopp \ + --with-ocf \ + +ifeq ($(BR2_PACKAGE_LIBFUSE),y) +CEPH_DEPENDENCIES += libfuse +CEPH_CONF_OPT += --with-fuse +else +CEPH_CONF_OPT += --without-fuse +endif + +ifeq ($(BR2_PACKAGE_LIBAIO),y) +CEPH_DEPENDENCIES += libaio +CEPH_CONF_OPT += --with-libaio +else +CEPH_CONF_OPT += --without-libaio +endif + +$(eval $(autotools-package))