From patchwork Fri Mar 9 19:17:15 2018 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Valentin Spreckels X-Patchwork-Id: 883955 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Authentication-Results: ozlabs.org; spf=none (mailfrom) smtp.mailfrom=lists.infradead.org (client-ip=2607:7c80:54:e::133; helo=bombadil.infradead.org; envelope-from=lede-dev-bounces+incoming=patchwork.ozlabs.org@lists.infradead.org; receiver=) Authentication-Results: ozlabs.org; dmarc=none (p=none dis=none) header.from=Informatik.Uni-Oldenburg.DE Authentication-Results: ozlabs.org; dkim=pass (2048-bit key; unprotected) header.d=lists.infradead.org header.i=@lists.infradead.org header.b="JMTk2uyr"; dkim-atps=neutral Received: from bombadil.infradead.org (bombadil.infradead.org [IPv6:2607:7c80:54:e::133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ozlabs.org (Postfix) with ESMTPS id 3zycZr6QVDz9scN for ; Sat, 10 Mar 2018 06:17:24 +1100 (AEDT) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20170209; h=Sender: Content-Transfer-Encoding:Content-Type:MIME-Version:List-Subscribe:List-Help: List-Post:List-Archive:List-Unsubscribe:List-Id:Subject:Message-Id:Date:To: From:Reply-To:Cc:Content-ID:Content-Description:Resent-Date:Resent-From: Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:In-Reply-To:References: List-Owner; bh=Ce5lDu99j0be0NEbWaKyMk5SV3gsyU4XoyKsRI+EI+4=; b=JMTk2uyr6xF95e 1suMN7/268IIc7gPGLllZBtHn7R3B1iHiWSXUFAdgsxFiCE9oc9h/HL/ciQNjrzRSyrebv2gr3dj7 LaqsV2LogAbNhkLcXRl2DkZyGL4jMm9oweuZzKbGtW+Nqq6+VstSyLJgJGMJa5OrAl6fqG57d7/4V 38fejjJMESyWU1OxvwelppBOXv5TiSmwEtICbfk/fzJ6kJu37+aMQlIVwQu6lu9/tu0BJ6GKxf2Ri 9yjRkCNPctxIAe0807QAcl8bO/HXvwICyjoUhYjdHC6j+Zqy61csuRyQJ9JpMWVZ3vnhbLf4aDCc5 eGjRbq6G5NVQOeV5bzvA==; Received: from localhost ([127.0.0.1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.89 #1 (Red Hat Linux)) id 1euNVx-0001Rh-DX; Fri, 09 Mar 2018 19:17:13 +0000 Received: from arbi.informatik.uni-oldenburg.de ([134.106.1.7]) by bombadil.infradead.org with esmtp (Exim 4.89 #1 (Red Hat Linux)) id 1euNVD-00010T-92 for lede-dev@lists.infradead.org; Fri, 09 Mar 2018 19:16:32 +0000 Received: from weser.Informatik.Uni-Oldenburg.DE ([134.106.11.83]) by arbi.Informatik.Uni-Oldenburg.DE (Exim 4.72) id 1euNV5-0007N7-PL; Fri, 09 Mar 2018 20:16:19 +0100 Received: from ahi.Informatik.Uni-Oldenburg.DE. ([134.106.9.61]) by weser.Informatik.Uni-Oldenburg.DE (Exim 4.80.1) id 1euNUx-000JWv-EU; Fri, 09 Mar 2018 20:16:11 +0100 From: "Valentin Spreckels" To: lede-dev@lists.infradead.org Date: Fri, 9 Mar 2018 20:17:15 +0100 Message-Id: <20180309191717.19545-1-Valentin.Spreckels@Informatik.Uni-Oldenburg.DE> X-Mailer: git-send-email 2.13.6 X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20180309_111628_191498_A5A1A4A1 X-CRM114-Status: GOOD ( 19.38 ) X-Spam-Score: -4.2 (----) X-Spam-Report: SpamAssassin version 3.4.1 on bombadil.infradead.org summary: Content analysis details: (-4.2 points) pts rule name description ---- ---------------------- -------------------------------------------------- -2.3 RCVD_IN_DNSWL_MED RBL: Sender listed at http://www.dnswl.org/, medium trust [134.106.1.7 listed in list.dnswl.org] -0.0 T_RP_MATCHES_RCVD Envelope sender domain matches handover relay domain -1.9 BAYES_00 BODY: Bayes spam probability is 0 to 1% [score: 0.0000] Subject: [LEDE-DEV] [PATCH 1/3] fritz-tools: add fritz_tffs_nand_read tool X-BeenThere: lede-dev@lists.infradead.org X-Mailman-Version: 2.1.21 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , MIME-Version: 1.0 Sender: "Lede-dev" Errors-To: lede-dev-bounces+incoming=patchwork.ozlabs.org@lists.infradead.org This tool reads informations from AVM TFFS 3.0 filesystems on nand flash. AVM uses this on newer FRITZ!Boxes (e.g. 7412) without SPI or nor flash. Signed-off-by: Valentin Spreckels --- package/utils/fritz-tools/Makefile | 15 + package/utils/fritz-tools/src/CMakeLists.txt | 3 +- .../utils/fritz-tools/src/fritz_tffs_nand_read.c | 558 +++++++++++++++++++++ 3 files changed, 575 insertions(+), 1 deletion(-) create mode 100644 package/utils/fritz-tools/src/fritz_tffs_nand_read.c diff --git a/package/utils/fritz-tools/Makefile b/package/utils/fritz-tools/Makefile index 7c547e6f97..a4d69bdf71 100644 --- a/package/utils/fritz-tools/Makefile +++ b/package/utils/fritz-tools/Makefile @@ -21,6 +21,15 @@ define Package/fritz-tffs/description Utility to partially read the TFFS filesystems. endef +define Package/fritz-tffs-nand + $(call Package/fritz-tools/Default) + TITLE:=Utility to partially read the TFFS filesystems on NAND flash +endef + +define Package/fritz-tffs-nand/description + Utility to partially read the TFFS filesystems on NAND flash. +endef + define Package/fritz-caldata $(call Package/fritz-tools/Default) DEPENDS:=+zlib @@ -36,10 +45,16 @@ define Package/fritz-tffs/install $(INSTALL_BIN) $(PKG_BUILD_DIR)/fritz_tffs_read $(1)/usr/bin/fritz_tffs endef +define Package/fritz-tffs-nand/install + $(INSTALL_DIR) $(1)/usr/bin + $(INSTALL_BIN) $(PKG_BUILD_DIR)/fritz_tffs_nand_read $(1)/usr/bin/fritz_tffs_nand +endef + define Package/fritz-caldata/install $(INSTALL_DIR) $(1)/usr/bin $(INSTALL_BIN) $(PKG_BUILD_DIR)/fritz_cal_extract $(1)/usr/bin/ endef $(eval $(call BuildPackage,fritz-tffs)) +$(eval $(call BuildPackage,fritz-tffs-nand)) $(eval $(call BuildPackage,fritz-caldata)) diff --git a/package/utils/fritz-tools/src/CMakeLists.txt b/package/utils/fritz-tools/src/CMakeLists.txt index 021dc7733a..85b815712e 100644 --- a/package/utils/fritz-tools/src/CMakeLists.txt +++ b/package/utils/fritz-tools/src/CMakeLists.txt @@ -9,7 +9,8 @@ FIND_PATH(zlib_include_dir zlib.h) INCLUDE_DIRECTORIES(${zlib_include_dir}) ADD_EXECUTABLE(fritz_tffs_read fritz_tffs_read.c) +ADD_EXECUTABLE(fritz_tffs_nand_read fritz_tffs_nand_read.c) ADD_EXECUTABLE(fritz_cal_extract fritz_cal_extract.c) TARGET_LINK_LIBRARIES(fritz_cal_extract z) -INSTALL(TARGETS fritz_tffs_read fritz_cal_extract RUNTIME DESTINATION bin) +INSTALL(TARGETS fritz_tffs_read fritz_tffs_nand_read fritz_cal_extract RUNTIME DESTINATION bin) diff --git a/package/utils/fritz-tools/src/fritz_tffs_nand_read.c b/package/utils/fritz-tools/src/fritz_tffs_nand_read.c new file mode 100644 index 0000000000..6f30c2bbdb --- /dev/null +++ b/package/utils/fritz-tools/src/fritz_tffs_nand_read.c @@ -0,0 +1,558 @@ +/* + * A tool for reading the TFFS partitions (a name-value storage usually + * found in AVM Fritz!Box based devices) on nand flash. + * + * Copyright (c) 2018 Valentin Spreckels + * + * Based on the fritz_tffs_read tool: + * Copyright (c) 2015-2016 Martin Blumenstingl + * and on the TFFS 2.0 kernel driver from AVM: + * Copyright (c) 2004-2007 AVM GmbH + * and the TFFS 3.0 kernel driver from AVM: + * Copyright (C) 2004-2014 AVM GmbH + * and the OpenWrt TFFS kernel driver: + * Copyright (c) 2013 John Crispin + * + * This program is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License as published by + * the Free Software Foundation; either version 2 of the License, or + * (at your option) any later version. + * + * This program is distributed in the hope that it will be useful, + * but WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License for more details. + * + * You should have received a copy of the GNU General Public License along + * with this program; if not, write to the Free Software Foundation, Inc., + * 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA. + */ + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#define DEFAULT_TFFS_SIZE (256 * 1024) + +#define TFFS_ID_END 0xffffffff +#define TFFS_ID_TABLE_NAME 0x000001ff + +#define TFFS_BLOCK_HEADER_MAGIC 0x41564d5f54464653ULL +#define TFFS_VERSION 0x0003 +#define TFFS_ENTRY_HEADER_SIZE 0x18 +#define TFFS_MAXIMUM_SEGMENT_SIZE (0x800 - TFFS_ENTRY_HEADER_SIZE) + +#define TFFS_SECTOR_SIZE 0x0800 +#define TFFS_SECTOR_OOB_SIZE 0x0040 +#define TFFS_SECTORS_PER_PAGE 2 + +#define TFFS_SEGMENT_CLEARED 0xffffffff + +static char *progname; +static char *mtddev; +static char *name_filter = NULL; +static bool show_all = false; +static bool print_all_key_names = false; +static bool swap_bytes = false; +static uint8_t readbuf[TFFS_SECTOR_SIZE]; +static uint8_t oobbuf[TFFS_SECTOR_OOB_SIZE]; +static uint32_t blocksize; +static int mtdfd; +struct tffs_sectors *sectors; + +struct tffs_sectors { + uint32_t num_sectors; + uint8_t sectors[0]; +}; + +static inline void sector_mark_bad(int num) { + sectors->sectors[num / 8] &= ~(0x80 >> (num % 8)); +}; + +static inline uint8_t sector_get_good(int num) { + return sectors->sectors[num / 8] & 0x80 >> (num % 8); +}; + +struct tffs_entry_segment { + uint32_t len; + void *val; +}; + +struct tffs_entry { + uint32_t len; + void *val; +}; + +struct tffs_name_table_entry { + uint32_t id; + char *val; +}; + +struct tffs_key_name_table { + uint32_t size; + struct tffs_name_table_entry *entries; +}; + +static inline uint8_t read_uint8(void *buf, ptrdiff_t off) { + return *(uint8_t *)(buf + off); +} + +static inline uint32_t read_uint32(void *buf, ptrdiff_t off) { + uint32_t tmp = *(uint32_t *)(buf + off); + if (swap_bytes) { + tmp = be32toh(tmp); + } + return tmp; +} + +static inline uint64_t read_uint64(void *buf, ptrdiff_t off) { + uint64_t tmp = *(uint64_t *)(buf + off); + if (swap_bytes) { + tmp = be64toh(tmp); + } + return tmp; +} + +static int read_sector(off_t pos) { + if (pread(mtdfd, readbuf, TFFS_SECTOR_SIZE, pos) != TFFS_SECTOR_SIZE) { + return -1; + } + + return 0; +} + +static int read_sectoroob(off_t pos) { + struct mtd_oob_buf oob = { + .start = pos, + .length = TFFS_SECTOR_OOB_SIZE, + .ptr = oobbuf + }; + + if (ioctl(mtdfd, MEMREADOOB, &oob) < 0) { + return -1; + } + + return 0; +} + +static inline uint32_t get_walk_size(uint32_t entry_len) +{ + return (entry_len + 3) & ~0x03; +} + +static void print_entry_value(const struct tffs_entry *entry) +{ + /* These are NOT NULL terminated. */ + fwrite(entry->val, 1, entry->len, stdout); +} + +static int find_entry(uint32_t id, struct tffs_entry *entry) +{ + uint32_t rev = 0; + uint32_t num_segments = 0; + struct tffs_entry_segment *segments = NULL; + + off_t pos = 0; + uint8_t block_ended = 0; + for (uint32_t sector = 0; sector < sectors->num_sectors; sector++, pos += TFFS_SECTOR_SIZE) { + if (block_ended) { + if (pos % blocksize == 0) { + block_ended = 0; + } + } else if (sector_get_good(sector)) { + if (read_sectoroob(pos) || read_sector(pos)) { + fprintf(stderr, "ERROR: sector isn't readable, but has been previously!\n"); + exit(EXIT_FAILURE); + } + uint32_t oob_id = read_uint32(oobbuf, 0x02); + uint32_t oob_len = read_uint32(oobbuf, 0x06); + uint32_t oob_rev = read_uint32(oobbuf, 0x0a); + uint32_t read_id = read_uint32(readbuf, 0x00); + uint32_t read_len = read_uint32(readbuf, 0x04); + uint32_t read_rev = read_uint32(readbuf, 0x0c); + if (oob_id != read_id || oob_len != read_len || oob_rev != read_rev) { + fprintf(stderr, "Warning: sector has inconsistent metadata\n"); + continue; + } + if (read_id == TFFS_ID_END) { + // no more entries in this block + block_ended = 1; + continue; + } + if (read_len > TFFS_MAXIMUM_SEGMENT_SIZE) { + fprintf(stderr, "Warning: segment is longer than possible\n"); + continue; + } + if (read_id == id) { + if (read_rev < rev) { + // obsolete revision => ignore this + continue; + } + if (read_rev > rev) { + // newer revision => clear old data + for (uint32_t i = 0; i < num_segments; i++) { + free(segments[i].val); + } + free (segments); + rev = read_rev; + num_segments = 0; + segments = NULL; + } + + uint32_t seg = read_uint32(readbuf, 0x10); + + if (seg == TFFS_SEGMENT_CLEARED) { + continue; + } + + uint32_t next_seg = read_uint32(readbuf, 0x14); + + uint32_t new_num_segs = next_seg == 0 ? seg + 1 : next_seg + 1; + if (new_num_segs > num_segments) { + segments = realloc(segments, new_num_segs * sizeof(struct tffs_entry_segment)); + memset(segments + (num_segments * sizeof(struct tffs_entry_segment)), 0x0, + (new_num_segs - num_segments) * sizeof(struct tffs_entry_segment)); + num_segments = new_num_segs; + } + segments[seg].len = read_len; + segments[seg].val = malloc(read_len); + memcpy(segments[seg].val, readbuf + TFFS_ENTRY_HEADER_SIZE, read_len); + } + } + } + + if (num_segments == 0) { + return 0; + } + + assert (segments != NULL); + + uint32_t len = 0; + for (uint32_t i = 0; i < num_segments; i++) { + if (segments[i].val == NULL) { + // missing segment + return 0; + } + + len += segments[i].len; + } + + void *p = malloc(len); + entry->val = p; + entry->len = len; + for (uint32_t i = 0; i < num_segments; i++) { + memcpy(p, segments[i].val, segments[i].len); + p += segments[i].len; + } + + return 1; +} + +static void parse_key_names(struct tffs_entry *names_entry, + struct tffs_key_name_table *key_names) +{ + uint32_t pos = 0, i = 0; + struct tffs_name_table_entry *name_item; + + key_names->entries = NULL; + + do { + key_names->entries = realloc(key_names->entries, + sizeof(struct tffs_name_table_entry) * (i + 1)); + if (key_names->entries == NULL) { + fprintf(stderr, "ERROR: memory allocation failed!\n"); + exit(EXIT_FAILURE); + } + name_item = &key_names->entries[i]; + + name_item->id = read_uint32(names_entry->val, pos); + pos += sizeof(uint32_t); + name_item->val = strdup((const char *)(names_entry->val + pos)); + + /* + * There is no "length" field because the string values are + * simply NULL-terminated -> strlen() gives us the size. + */ + pos += get_walk_size(strlen(name_item->val) + 1); + + ++i; + } while (pos < names_entry->len); + + key_names->size = i; +} + +static void show_all_key_names(struct tffs_key_name_table *key_names) +{ + for (uint32_t i = 0; i < key_names->size; i++) + printf("%s\n", key_names->entries[i].val); +} + +static int show_all_key_value_pairs(struct tffs_key_name_table *key_names) +{ + uint8_t has_value = 0; + struct tffs_entry tmp; + + for (uint32_t i = 0; i < key_names->size; i++) { + if (find_entry(key_names->entries[i].id, &tmp)) { + printf("%s=", (const char *)key_names->entries[i].val); + print_entry_value(&tmp); + printf("\n"); + has_value++; + free(tmp.val); + } + } + + if (!has_value) { + fprintf(stderr, "ERROR: no values found!\n"); + return EXIT_FAILURE; + } + + return EXIT_SUCCESS; +} + +static int show_matching_key_value(struct tffs_key_name_table *key_names) +{ + struct tffs_entry tmp; + const char *name; + + for (uint32_t i = 0; i < key_names->size; i++) { + name = key_names->entries[i].val; + + if (strncmp(name, name_filter, strlen(name)) == 0) { + if (find_entry(key_names->entries[i].id, &tmp)) { + print_entry_value(&tmp); + printf("\n"); + free(tmp.val); + return EXIT_SUCCESS; + } else { + fprintf(stderr, + "ERROR: no value found for name %s!\n", + name); + return EXIT_FAILURE; + } + } + } + + fprintf(stderr, "ERROR: Unknown key name %s!\n", name_filter); + return EXIT_FAILURE; +} + +static int check_sector(off_t pos) +{ + if (read_sectoroob(pos)) { + return 0; + } + if (read_uint8(oobbuf, 0x00) != 0xff) { + // block is bad + return 0; + } + if (read_uint8(oobbuf, 0x01) != 0xff) { + // sector is bad + return 0; + } + return 1; +} + +static int check_block(off_t pos, uint32_t sector) +{ + if (!check_sector(pos)) { + return 0; + } + if (read_sector(pos)) { + return 0; + } + if (read_uint64(readbuf, 0x00) != TFFS_BLOCK_HEADER_MAGIC) { + fprintf(stderr, "Warning: block without magic header. Skipping block\n"); + return 0; + } + if (read_uint32(readbuf, 0x0c) != TFFS_SECTORS_PER_PAGE) { + fprintf(stderr, "Warning: block with wrong number of sectors per page. Skipping block\n"); + return 0; + } + + uint32_t num_hdr_bad = read_uint32(readbuf, 0x0c); + for (uint32_t i = 0; i < num_hdr_bad; i++) { + uint32_t bad = sector + read_uint64(readbuf, 0x1c + sizeof(uint64_t)*i); + sector_mark_bad(bad); + } + + return 1; +} + +static int scan_mtd() +{ + struct mtd_info_user info; + + if (ioctl(mtdfd, MEMGETINFO, &info)) { + return 0; + } + + blocksize = info.erasesize; + + sectors = malloc(sizeof(*sectors) + (info.size / TFFS_SECTOR_SIZE + 7) / 8); + if (sectors == NULL) { + fprintf(stderr, "ERROR: memory allocation failed!\n"); + exit(EXIT_FAILURE); + } + sectors->num_sectors = info.size / TFFS_SECTOR_SIZE; + memset(sectors->sectors, 0xff, (info.size / TFFS_SECTOR_SIZE + 7) / 8); + + uint32_t sector = 0, valid_blocks = 0; + uint8_t block_ok = 0; + for (off_t pos = 0; pos < info.size; sector++, pos += TFFS_SECTOR_SIZE) { + if (pos % info.erasesize == 0) { + block_ok = check_block(pos, sector); + // first sector of the block contains metadata + // => handle it like a bad sector + sector_mark_bad(sector); + if (block_ok) { + valid_blocks++; + } + } else if (!block_ok || !sector_get_good(sector) || !check_sector(pos)) { + sector_mark_bad(sector); + } + } + + return valid_blocks; +} + +static void usage(int status) +{ + FILE *stream = (status != EXIT_SUCCESS) ? stderr : stdout; + + fprintf(stream, "Usage: %s [OPTIONS...]\n", progname); + fprintf(stream, + "\n" + "Options:\n" + " -a list all key value pairs found in the TFFS file/device\n" + " -d inspect the TFFS on mtd device \n" + " -h show this screen\n" + " -l list all supported keys\n" + " -n display the value of the given key\n" + ); + + exit(status); +} + +static void parse_options(int argc, char *argv[]) +{ + while (1) + { + int c; + + c = getopt(argc, argv, "abd:hln:"); + if (c == -1) + break; + + switch (c) { + case 'a': + show_all = true; + name_filter = NULL; + print_all_key_names = false; + break; + case 'b': + swap_bytes = 1; + break; + case 'd': + mtddev = optarg; + break; + case 'h': + usage(EXIT_SUCCESS); + break; + case 'l': + print_all_key_names = true; + show_all = false; + name_filter = NULL; + break; + case 'n': + name_filter = optarg; + show_all = false; + print_all_key_names = false; + break; + default: + usage(EXIT_FAILURE); + break; + } + } + + if (!mtddev) { + fprintf(stderr, "ERROR: No input file (-d ) given!\n"); + usage(EXIT_FAILURE); + } + + if (!show_all && !name_filter && !print_all_key_names) { + fprintf(stderr, + "ERROR: either -l, -a or -n is required!\n"); + usage(EXIT_FAILURE); + } +} + +int main(int argc, char *argv[]) +{ + int ret = EXIT_FAILURE; + struct tffs_entry name_table; + struct tffs_key_name_table key_names; + + progname = basename(argv[0]); + + parse_options(argc, argv); + + mtdfd = open(mtddev, O_RDONLY); + if (mtdfd < 0) { + fprintf(stderr, "ERROR: Failed to open tffs device %s\n", + mtddev); + goto out; + } + + if (!scan_mtd()) { + fprintf(stderr, "ERROR: Parsing blocks from tffs device %s failed\n", mtddev); + fprintf(stderr," Is byte-swapping (-b) required?\n"); + goto out_close; + } + + if (!find_entry(TFFS_ID_TABLE_NAME, &name_table)) { + fprintf(stderr,"ERROR: No name table found on tffs device %s\n", + mtddev); + goto out_free_sectors; + } + + parse_key_names(&name_table, &key_names); + if (key_names.size < 1) { + fprintf(stderr, "ERROR: No name table found on tffs device %s\n", + mtddev); + goto out_free_entry; + } + + if (print_all_key_names) { + show_all_key_names(&key_names); + ret = EXIT_SUCCESS; + } else if (show_all) { + ret = show_all_key_value_pairs(&key_names); + } else { + ret = show_matching_key_value(&key_names); + } + + free(key_names.entries); +out_free_entry: + free(name_table.val); +out_free_sectors: + free(sectors); +out_close: + close(mtdfd); +out: + return ret; +}