From patchwork Wed Jan 4 21:06:30 2012 Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit X-Patchwork-Submitter: Chuck Lever X-Patchwork-Id: 134359 Return-Path: X-Original-To: incoming@patchwork.ozlabs.org Delivered-To: patchwork-incoming@bilbo.ozlabs.org Received: from rcsinet15.oracle.com (rcsinet15.oracle.com [148.87.113.117]) (using TLSv1 with cipher DHE-RSA-AES256-SHA (256/256 bits)) (Client CN "rcsinet15.oracle.com", Issuer "VeriSign Class 3 International Server CA - G3" (verified OK)) by ozlabs.org (Postfix) with ESMTPS id 1DB461007D7 for ; Thu, 5 Jan 2012 08:06:45 +1100 (EST) Received: from acsinet22.oracle.com (acsinet22.oracle.com [141.146.126.238]) by rcsinet15.oracle.com (Switch-3.4.4/Switch-3.4.4) with ESMTP id q04L6anE032125 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=OK); Wed, 4 Jan 2012 21:06:37 GMT Received: from oss.oracle.com (oss.oracle.com [141.146.12.120]) by acsinet22.oracle.com (8.14.4+Sun/8.14.4) with ESMTP id q04L6ZKJ006383 (version=TLSv1/SSLv3 cipher=DHE-RSA-AES256-SHA bits=256 verify=NO); Wed, 4 Jan 2012 21:06:36 GMT Received: from localhost ([127.0.0.1] helo=oss.oracle.com) by oss.oracle.com with esmtp (Exim 4.63) (envelope-from ) id 1RiY2d-0001gX-Pr; Wed, 04 Jan 2012 13:06:35 -0800 Received: from acsinet13.oracle.com ([141.146.126.235]) by oss.oracle.com with esmtp (Exim 4.63) (envelope-from ) id 1RiY2b-0001gQ-Q2 for fedfs-utils-devel@oss.oracle.com; Wed, 04 Jan 2012 13:06:33 -0800 Received: from mail-yx0-f171.google.com (mail-yx0-f171.google.com [209.85.213.171]) by acsinet13.oracle.com (Switch-3.4.4/Switch-3.4.4) with ESMTP id q04L6Voq016717 (version=TLSv1/SSLv3 cipher=RC4-SHA bits=128 verify=FAIL) for ; Wed, 4 Jan 2012 21:06:33 GMT Received: by yenr9 with SMTP id r9so9965234yen.2 for ; Wed, 04 Jan 2012 13:06:31 -0800 (PST) Received: by 10.101.208.1 with SMTP id k1mr9608512anq.12.1325711191680; Wed, 04 Jan 2012 13:06:31 -0800 (PST) Received: from degas.1015granger.net (adsl-99-26-161-222.dsl.sfldmi.sbcglobal.net. [99.26.161.222]) by mx.google.com with ESMTPS id 16sm55523240ank.7.2012.01.04.13.06.30 (version=TLSv1/SSLv3 cipher=OTHER); Wed, 04 Jan 2012 13:06:31 -0800 (PST) From: Chuck Lever To: fedfs-utils-devel@oss.oracle.com Date: Wed, 04 Jan 2012 16:06:30 -0500 Message-ID: <20120104210629.8810.39328.stgit@degas.1015granger.net> In-Reply-To: <20120104204207.8810.10569.stgit@degas.1015granger.net> References: <20120104204207.8810.10569.stgit@degas.1015granger.net> User-Agent: StGIT/0.14.3 MIME-Version: 1.0 X-Flow-Control-Info: class=Default ip=209.85.213.171 ct-class=G1 ct-vol1=0 ct-vol2=0 ct-vol3=0 ct-risk=0 ct-spam1=0 ct-spam2=0 ct-bulk=0 rcpts=1 size=7333 Subject: [fedfs-utils] [PATCH 2/7] fedfsd: Flush kernel's exports cache after junction operations X-BeenThere: fedfs-utils-devel@oss.oracle.com X-Mailman-Version: 2.1.9 Precedence: list Reply-To: fedfs-utils Developers List-Id: fedfs-utils Developers List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: fedfs-utils-devel-bounces@oss.oracle.com Errors-To: fedfs-utils-devel-bounces@oss.oracle.com X-Source-IP: acsinet22.oracle.com [141.146.126.238] X-CT-RefId: str=0001.0A090208.4F04BF5D.0115:SCFSTAT1119972, ss=1, re=-6.300, fgs=0 X-Auth-Type: Internal IP We want the effects of junction operations to appear to NFS clients immediately. But NFSD's export cache will hang onto old junction information for a time. All we can do is flush that cache after any operation that changes a junction. FedFS ADMIN clients can explicitly request cache flushing by using certain parameters to the FEDFS_LOOKUP_JUNCTION operation. Introduce an API to libjunction that can request an export cache flush. Call that function in appropriate places within fedfsd. This is cribbed from the exportfs command in nfs-utils. Signed-off-by: Chuck Lever --- src/fedfsd/svc.c | 25 ++++++-- src/include/junction.h | 2 + src/libjunction/Makefile.am | 2 - src/libjunction/export-cache.c | 121 ++++++++++++++++++++++++++++++++++++++++ 4 files changed, 141 insertions(+), 9 deletions(-) create mode 100644 src/libjunction/export-cache.c diff --git a/src/fedfsd/svc.c b/src/fedfsd/svc.c index 316042f..6d548c1 100644 --- a/src/fedfsd/svc.c +++ b/src/fedfsd/svc.c @@ -538,15 +538,16 @@ fedfsd_svc_create_junction_1(SVCXPRT *xprt) } result = fedfs_store_fsn(pathname, fsn_uuid, host); - if (result != FEDFS_OK) + if (result != FEDFS_OK) { xlog(D_GENERAL, "%s: fedfs_store_fsn", __func__); - else { - xlog(D_CALL, "%s: uuid: %s", - __func__, fsn_uuid); - xlog(D_CALL, "%s: nsdb: %s:%u", - __func__, nsdb_hostname(host), nsdb_port(host)); + goto out; } + (void)junction_flush_exports_cache(); + xlog(D_CALL, "%s: uuid: %s", __func__, fsn_uuid); + xlog(D_CALL, "%s: nsdb: %s:%u", + __func__, nsdb_hostname(host), nsdb_port(host)); + out: xlog(D_CALL, "%s: Replying with %s", __func__, nsdb_display_fedfsstatus(result)); @@ -643,6 +644,7 @@ fedfsd_svc_delete_junction_1(SVCXPRT *xprt) goto out; fedfsd_rmdir(pathname); + (void)junction_flush_exports_cache(); result = FEDFS_OK; out: @@ -800,9 +802,11 @@ fedfsd_svc_lookup_junction_1(SVCXPRT *xprt) result.status = FEDFS_ERR_BADXDR; switch (args.resolve) { case FEDFS_RESOLVE_NONE: - case FEDFS_RESOLVE_CACHE: case FEDFS_RESOLVE_NSDB: break; + case FEDFS_RESOLVE_CACHE: + result.status = FEDFS_ERR_UNKNOWN_CACHE; + goto out; default: goto out; } @@ -839,7 +843,6 @@ fedfsd_svc_lookup_junction_1(SVCXPRT *xprt) switch (args.resolve) { case FEDFS_RESOLVE_NONE: - case FEDFS_RESOLVE_CACHE: break; case FEDFS_RESOLVE_NSDB: result.status = nsdb_open_nsdb(host, NULL, NULL, &ldap_err); @@ -857,6 +860,12 @@ fedfsd_svc_lookup_junction_1(SVCXPRT *xprt) break; result.status = fedfsd_prepare_fedfsfsl_array(fsls, resok); nsdb_free_fedfs_fsls(fsls); + if (result.status != FEDFS_OK) + break; + result.status = junction_flush_exports_cache(); + if (result.status != FEDFS_OK && + result.status != FEDFS_ERR_NO_CACHE_UPDATE) + result.status = FEDFS_OK; break; default: result.status = FEDFS_ERR_SVRFAULT; diff --git a/src/include/junction.h b/src/include/junction.h index e09cf5b..edbab7b 100644 --- a/src/include/junction.h +++ b/src/include/junction.h @@ -39,4 +39,6 @@ FedFsStatus fedfs_is_junction(const char *pathname); FedFsStatus fedfs_save_mode(const char *pathname); FedFsStatus fedfs_restore_mode(const char *pathname); +FedFsStatus junction_flush_exports_cache(void); + #endif /* !_FEDFS_JUNCTION_H_ */ diff --git a/src/libjunction/Makefile.am b/src/libjunction/Makefile.am index bc97bf3..4c43e66 100644 --- a/src/libjunction/Makefile.am +++ b/src/libjunction/Makefile.am @@ -24,7 +24,7 @@ ## noinst_LTLIBRARIES = libjunction.la -libjunction_la_SOURCES = junction.c +libjunction_la_SOURCES = export-cache.c junction.c CLEANFILES = cscope.in.out cscope.out cscope.po.out *~ DISTCLEANFILES = Makefile.in diff --git a/src/libjunction/export-cache.c b/src/libjunction/export-cache.c new file mode 100644 index 0000000..e344547 --- /dev/null +++ b/src/libjunction/export-cache.c @@ -0,0 +1,121 @@ +/** + * @file src/libjunction/export-cache.c + * @brief Try to flush NFSD's exports cache + */ + +/* + * Copyright 2011 Oracle. All rights reserved. + * + * This file is part of fedfs-utils. + * + * fedfs-utils is free software; you can redistribute it and/or modify + * it under the terms of the GNU General Public License version 2.0 as + * published by the Free Software Foundation. + * + * fedfs-utils is distributed in the hope that it will be useful, but + * WITHOUT ANY WARRANTY; without even the implied warranty of + * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the + * GNU General Public License version 2.0 for more details. + * + * You should have received a copy of the GNU General Public License + * version 2.0 along with fedfs-utils. If not, see: + * + * http://www.gnu.org/licenses/old-licenses/gpl-2.0.txt + */ + +#include +#include + +#include +#include +#include +#include +#include +#include +//#include +#include + + +#include "fedfs.h" +#include "nsdb.h" +#include "junction.h" +#include "xlog.h" + +/** + * Ordered list of proc files to poke when requesting an NFSD cache flush + */ +static const char *junction_proc_files[] = { + "/proc/net/rpc/auth.unix.ip/flush", + "/proc/net/rpc/auth.unix.gid/flush", + "/proc/net/rpc/nfsd.fh/flush", + "/proc/net/rpc/nfsd.export/flush", + NULL, +}; + +/** + * Write time into one file + * + * @param pathname NUL-terminated C string containing POSIX pathname of file to write + * @param flushtime NUL-terminated C string containing current time in seconds since the Epoch + * @return a FedFsStatus code + */ +static FedFsStatus +junction_write_time(const char *pathname, const char *flushtime) +{ + FedFsStatus retval; + ssize_t len; + int fd; + + fd = open(pathname, O_RDWR); + if (fd == -1) { + xlog(D_GENERAL, "%s: Failed to open %s: %m", + __func__, pathname); + /* If the proc files don't exist, no server + * is running on this system */ + return FEDFS_ERR_NO_CACHE_UPDATE; + } + + len = write(fd, flushtime, strlen(flushtime)); + if (len != (ssize_t)strlen(flushtime)) { + xlog(D_GENERAL, "%s: Failed to write %s: %m", + __func__, pathname); + /* If the proc files exist but the update failed, + * we don't know the state of the cache */ + retval = FEDFS_ERR_UNKNOWN_CACHE; + } else + /* Cache flush succeeded */ + retval = FEDFS_OK; + + (void)close(fd); + return retval; +} + +/** + * Flush the kernel NFSD's exports cache + * + * @return a FedFsStatus code + */ +FedFsStatus +junction_flush_exports_cache(void) +{ + FedFsStatus retval; + char flushtime[20]; + unsigned int i; + time_t now; + + xlog(D_CALL, "%s: Flushing NFSD caches...", __func__); + + now = time(NULL); + if (now == -1) { + xlog(D_GENERAL, "%s: time(3) failed", __func__); + return FEDFS_ERR_SVRFAULT; + } + snprintf(flushtime, sizeof(flushtime), "%ld\n", now); + + for (i = 0; junction_proc_files[i] != NULL; i++) { + retval = junction_write_time(junction_proc_files[i], flushtime); + if (retval != FEDFS_OK) + return retval; + } + return FEDFS_OK; +}