From patchwork Sun Jul 3 09:29:54 2016 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Luke McKee X-Patchwork-Id: 643714 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from bombadil.infradead.org (bombadil.infradead.org [IPv6:2001:1868:205::9]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3rj4hN35txz9t0h for ; Sun, 3 Jul 2016 19:33:23 +1000 (AEST) Authentication-Results: ozlabs.org; dkim=fail reason="signature verification failed" (2048-bit key; unprotected) header.d=gmail.com header.i=@gmail.com header.b=sjKigjK6; dkim-atps=neutral Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.85_2 #1 (Red Hat Linux)) id 1bJdjJ-0001Ab-ID; Sun, 03 Jul 2016 09:30:21 +0000 Received: from mail-pf0-x233.google.com ([2607:f8b0:400e:c00::233]) by bombadil.infradead.org with esmtps (Exim 4.85_2 #1 (Red Hat Linux)) id 1bJdjH-0008Op-DY for lede-dev@lists.infradead.org; Sun, 03 Jul 2016 09:30:20 +0000 Received: by mail-pf0-x233.google.com with SMTP id c2so52968434pfa.2 for ; Sun, 03 Jul 2016 02:29:58 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=subject:to:references:from:message-id:date:user-agent:mime-version :in-reply-to:content-transfer-encoding; bh=pvRpalxgZA7/z3xdFlIUXYoYmRn4IBmXhowtjbfuG1k=; b=sjKigjK6t2s6fSiP4qv1292rMbs9mXS1CFTLboxM+ue2iPYfYuph7QNsMAwzuOEzP9 yKv2SeqTvXYtPR6ikQOScj1Iw6MrLDCaJrflmLTGM5prlgkeRapx9Ip4NsauOfhtlcrW 2tgge+LOCSokHuAKnaau6y04EgMLQ9JioSZOexO+Us6JoXSqOcIc7K6caKYB2XOzZzJk i97VwlqDt5bR6dWLa5hFMwBTo4IR3lTJGmAWLCRHtUtPKL/Y0zUd8butSsUfkAUo/pc8 x7GG4NBD3szk7SqFXitHKFeajg8qBn8cNYKAv9VXm497YTwt/Sj6Y+8+wlGrqyAf6E1j Vnow== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20130820; h=x-gm-message-state:subject:to:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-transfer-encoding; bh=pvRpalxgZA7/z3xdFlIUXYoYmRn4IBmXhowtjbfuG1k=; b=S8gT4BE7NfbiKlTFCo+FVe+oXtv2GgPa9kBheXqsr4cnAMDUku8Yp5FF8hMqpQjghk 1Y6/3UJZQzjCcafbG6SME9cRCnZ0eHemFx1eA7i36uAPvqzW4KCIwgjXA2OXuAIUCC4H ziI3NqZcZJdkOBSWJlRYQ/xecRM8cvuDqsOifQbf53HlzEuab/l5mAg6nzBStvZTmYDf 5AaxSFh9shI5REXpS76AFu5ORnNUPyWNTrbXm55Cx8X4upqnYKKayREvIEAya3wWUjw1 XFRbUYidUcnttac6jkVmXWM5512fdx8SdHrbHy7/o0QeX6iDvxy98xdQWOytUY+lMhMz dwjw== X-Gm-Message-State: ALyK8tIv41vZC/64zuTNgtOu/Q5eMawjwJp4S7MmzzT0UHyYvu5lzG+dNH1vY2gU0CV/TA== X-Received: by 10.98.155.29 with SMTP id r29mr12584614pfd.11.1467538197961; Sun, 03 Jul 2016 02:29:57 -0700 (PDT) Received: from [172.16.16.131] ([117.1.31.187]) by smtp.gmail.com with ESMTPSA id x67sm2026539pfa.56.2016.07.03.02.29.56 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Sun, 03 Jul 2016 02:29:57 -0700 (PDT) To: John Crispin , openwrt-devel@lists.openwrt.org, lede-dev@lists.infradead.org, arjen+lede@de-korte.org References: <056418e9-a364-e8d5-dff8-267ff237fffc@phrozen.org> From: Luke McKee Message-ID: Date: Sun, 3 Jul 2016 16:29:54 +0700 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:45.0) Gecko/20100101 Thunderbird/45.1.1 MIME-Version: 1.0 In-Reply-To: <056418e9-a364-e8d5-dff8-267ff237fffc@phrozen.org> X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20160703_023019_527028_D07FCCFE X-CRM114-Status: GOOD ( 22.57 ) X-Spam-Score: -2.7 (--) X-Spam-Report: SpamAssassin version 3.4.0 on bombadil.infradead.org summary: Content analysis details: (-2.7 points) pts rule name description ---- ---------------------- -------------------------------------------------- -0.7 RCVD_IN_DNSWL_LOW RBL: Sender listed at http://www.dnswl.org/, low trust [2607:f8b0:400e:c00:0:0:0:233 listed in] [list.dnswl.org] -0.0 SPF_PASS SPF: sender matches SPF record 0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail provider (hojuruku[at]gmail.com) -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1% [score: 0.0000] -0.1 DKIM_VALID Message has at least one valid DKIM or DK signature -0.1 DKIM_VALID_AU Message has a valid DKIM or DK signature from author's domain 0.1 DKIM_SIGNED Message has a DKIM or DK signature, not necessarily valid Subject: [LEDE-DEV] [PATCH V3] Re: procd: patch to support busybox mkfs.ext2 X-BeenThere: lede-dev@lists.infradead.org X-Mailman-Version: 2.1.20 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "Lede-dev" Errors-To: lede-dev-bounces+incoming=patchwork.ozlabs.org@lists.infradead.org On 07/03/2016 01:24 PM, John Crispin wrote: > Hi, > > few comments > > 1) the patch needs to be split into 2, ext2 change/lzo change > 2) send them inline please and not as attachement > 3) the V3 needs to be in the subject prefix [PATCH V3] procd: ... > 4) the Signed-off-by: line is missing > > John Hi John, 1) no can do. You are going to merge it into your project git page anyway, unless openwrt locked you out of your own project you are a maintainer of on git. In which case reply with the rebuke to .... 2) ok I fired up thunderbird, google mail destroys patches in plain text mode 3) done 4) done Don't blame me for the long email. I thought I was already done earlier but ..... I have tested this with lede trunk with v2 of the patch. V3 is the same just with a #ifdef, lz4 (faster less compression). It definitely works. It has 4mb flash. 140k of text files generated in my setup and placed in /tmp turned into 7.1k - neat! - however tmpfs has garbage collection zram doesn't exit unless you create a cron job that will dd if=/dev/zero bs=4096 of=/tmp/zero.bin count=sizeofreespace; rm /tmp/zero.bin. Deleting a file from /tmp doesn't allow memory reclamation like with tmpfs. Under normal use it will take a long time for this to really fill up, especially if ext2 is over-writing the same files & inodes. Zram module estimates there is 28meg of /tmp storage which is more than you'd get with tmpfs. My tp-link has 32mb ram 28 usable. Secure deletion on ext4 was never popular: https://lwn.net/Articles/462437/ Also of NOTE is tmpfs is STILL getting mounted. People with this feature on could in theory disable tmpfs. John you might want to look into disabling the tmpfs mount of ZRAM_TMPFS mounted instead. John: Constraints for the Makefile for procd to go with this I didn't submit, because I'm not sure config stanzas like package stanzas can have DEPENDS. Maybe you have to make it a fake package or something that really just passes on the defines to gcc so you can set up package dependencies. long zramsize; pid_t pid; @@ -94,6 +95,15 @@ } mkdev("*", 0600); +#ifdef ZRAM_TMPFS_LZ4 + fp = fopen("/sys/block/zram0/comp_algorithm", "r+"); + if (fp == NULL) { + ERROR("Can't open /sys/block/zram0/comp_algorithm: %s\n", strerror(errno)); + return errno; + } + fprintf(fp, "%s", "lz4" ); + fclose(fp); +#endif zramsize = proc_meminfo() / 2; fp = fopen("/sys/block/zram0/disksize", "r+"); @@ -107,10 +116,10 @@ pid = fork(); if (!pid) { execvp(mkfs[0], mkfs); - ERROR("Can't exec /sbin/mkfs.ext4\n"); + ERROR("Can't exec /sbin/mkfs.ext2\n"); exit(-1); } else if (pid <= 0) { - ERROR("Can't exec /sbin/mkfs.ext4\n"); + ERROR("Can't exec /sbin/mkfs.ext2\n"); return -1; } else { waitpid(pid, NULL, 0); ===================================== 1) Thou shall turn on CONFIG_BUSYBOX_CONFIG_MKFS_EXT2=y Justification: ~15-20k binary vs 350k (with stripping) for mkfs.ext2 and if you want the proper mkfs.ext4 etc you can install it in your overlay - extroot. Busybox mkfs.ext2 and e2fsprogs are configured not to collide with each-other. This patch makes the need for e2fsprogs libext2 redundant. Not having this (like we do right now) will flood the bug reports with idiots (like me) wondering why they bricked their router when it can't mount /tmp. 2) Debatable: Thou shall disable zram-swap, zram-swap shall block this PROC_ZRAM_TMPFS option. Justification: Right now they both want to use /dev/zram0, but the system will have no space for running apps if they are both running at the same time. You need to allocate 1/3 ram to each at best. That will have be a new compile time option in a further patch, that I will release should someone make the argument for wanting both of these. You either want 1 of 3 possible setups. #1 This patch. Best for people who need to opkg -d ram a lot of stuff #2 tmpfs and progs BOTH can use zram-swap which makes it a better overall choice, after thinking about this patch it isn't worth it as much as just using zram-swap. But better have a working feature than a broken one as it exists in buildroot now Or #3 Assuming you have a fast storage device connected to your openwrt (most don't but some NASes do) ZSWAP kernel will dynamically use free memory to offer a pre-swap cache, and allow compressed data (lzo/lz4) to be wrote to swap. It may require an actual initialized swap device to work. I haven't RTFMed yet. 3) Debatable: LZ4 should be on by default. (My two cents) Justification: It's almost compresses well as LZO but it's less CPU intensive for those without Floating point SMID's which is most home routers. So put some help about that in the Makefile. LZ4 is included as a dependency to the zram.ko module and will be loaded at early boot by modprobe / procd. Contrary to rumors circulating on openwrt's trac it actually works :) https://dev.openwrt.org/ticket/22666#comment:3 4) Highly recommended: Makefile should block option if your kernel isn't 4.4. Justification: https://dev.openwrt.org/ticket/21705 & above 5) Stop openwrt folks trying to patch the kernel to make two zram devices by default because you can hot-add them. https://dev.openwrt.org/ticket/19586#comment:10 Here's the patch --------------------------------------------- procd: remove dependency on bloated e2fsprogs/libext2 and use CONFIG_BUSYBOX_CONFIG_MKFS_EXT2=y At a minimum (see accompanying email on mailing list) advising my two cents on the pros and cons of this patch you need to add a configuration option to the Makefile if you want lz4 support config PROCD_ZRAM_TMPFS bool default n prompt "Mount /tmp using zram." endmenu & ifeq ($(CONFIG_PROCD_ZRAM_TMPFS_LZ4),y) CMAKE_OPTIONS += -DZRAM_TMPFS_LZ4=1 endif Signed-off-by: Luke McKee --- a/initd/zram.c 2016-07-03 06:39:51.011999930 +0700 +++ b/initd/zram.c 2016-07-03 07:00:34.143492847 +0700 @@ -82,7 +82,7 @@ int mount_zram_on_tmp(void) { - char *mkfs[] = { "/usr/sbin/mkfs.ext4", "-b", "4096", "-F", "-L", "TEMP", "-m", "0", "/dev/zram0", NULL }; + char *mkfs[] = { "/sbin/mkfs.ext2", "-b", "4096", "-F", "-L", "TEMP", "-m", "0", "/dev/zram0", NULL }; FILE *fp;