| Message ID | 20120827092801.31035.22348.stgit@perseus.themaw.net |
|---|---|
| State | Superseded |
| Headers | show |
Hi Ian- FYI, we've got a patchwork repository, your patch is now here: http://patchwork.ozlabs.org/patch/180159/ And I marked it "Under Review." Thanks for tackling this. Overall, the organization is right, but the details need a little work, and hopefully I've given you some pointers in the right direction. Also there is an INSTALL file in the tar ball. Should we augment that instead? On Aug 27, 2012, at 2:28 AM, Ian Kent wrote: > Add a howto to setup a DNS domain fedfs.org, an (OpenLDAP) LDAP > instance with context dc=fedfs,dc=org and work through the steps > to setup a read-only FedFS root domain with a single referral. > > Signed-off-by: Ian Kent <raven@themaw.net> > --- > > doc/howto/fedfs-simple-setup-howto.txt | 465 ++++++++++++++++++++++++++++++++ > 1 files changed, 465 insertions(+), 0 deletions(-) > create mode 100644 doc/howto/fedfs-simple-setup-howto.txt > > diff --git a/doc/howto/fedfs-simple-setup-howto.txt b/doc/howto/fedfs-simple-setup-howto.txt > new file mode 100644 > index 0000000..6454f64 > --- /dev/null > +++ b/doc/howto/fedfs-simple-setup-howto.txt > @@ -0,0 +1,465 @@ > + > +Simple example of basic FedFS setup and operation > +================================================= > + > +The examples in this test procedure assume a Fedora 18 install. > + > +Assume that the package fedfs-utils-server and fedfs-utils-client > +are installed on perseus.fedfs.org. And that fedfs-utils-server and > +a nfs-utils built with fedfs-utils-devel installed on zeus.fedfs.org > +and that fedfs-utils-lib is also installed on zeus.fedfs.org. Since fedfs-utils is nominally the "upstream" version, we should discuss the tar ball installation procedure here, and not a particular distributor's package installation. > + > +We want to setup perseus.fedfs.org to provide the domain root, > +be able to use FedFS to mount the domain root, and to setup and > +use a FedFS juntion to access an export on zeus.fedfs.org upon > +access to a directory within the domain root. And lastly, setup > +autofs to use the FedFS program map to mount the root of the > +domain. There are five roles in a FedFS domain: file server, filesystem client, NSDB, NSDB admin client, and DNS server. I find that helps organize the set-up steps. It's not always clear below where you are switching between perseus and zeus, nor which host is running the forwarding DNS or the LDAP service. It would be good to clarify that. But also I think at least the filesystem client (which mounts the domain root or which runs autofs) ought to be a separate host. > + > +#Questions still to be answered: > +#1. When and where (ie. on which server) is rpc.fedfsd used in the > +# procedure below. rpc.fedfsd is set up on the NFS file servers in the domain, including the file server that is sharing the domain root. Note, though, that it's optional: you can do the same operations by running "nfsref" or "nsdbparams" directly on file servers. > +#2. What is the procedure for setting up a junction on a third > +# unrelated machine. Works pretty much like setting up the domain root. Pick an export on some file server, and use "nfsref" to set up a junction. > + > +Setup a forwarding DNS server for fedfs.org > +=========================================== > + > +Some of this setup may not be necessary but it is what I use to setup > +DNS instances for testsing. Also many of the files created in step 1 > +should be included in a caching nameserver package. Installing the > +cacheing nameserver package and making appropriate modifications to > +turn the cacheing nameserver into a forwarding nameserver iis really > +all that needs to be done. Of course the fedfs zone files will always > +be needed. > + > +The example assumes the private IP address range 192.168.1 is being > +used and a nameserver that is able to satisfy name resolution for > +external names is at 192.168.1.1 and the hosts perseus.fedfs.org and > +zeus.fedfs.org are assigned addresses 192.168.1.32 and 192.168.1.31 > +respectively. > + > +1. Setup zeus.fedfs.org as a forwarding DNS server: Using a forwarding server is an interesting choice… we often do it this way for testing. But the point of the SRV record is to advertise the domain root globally. Should we describe here how to add an SRV to a non-forwarding DNS server instead, or in addition to, the below recipe? Ah, much farther along, you set up the SRV record. > + > + mkdir -p /var/named/master/fedfs.org > + if [ ! -d /var/named/data ]; then > + mkdir /var/named/data > + fi > + > +Create /var/named/named.root using: > + dig +norec NS > /var/named/named.root > + > +Create /etc/named.rfc1912.zones as: > + // named.rfc1912.zones: > + // > + // ISC BIND named zone configuration for zones recommended by > + // RFC 1912 section 4.1 : localhost TLDs and address zones > + // > + zone "localdomain" IN { > + type master; > + file "localdomain.zone"; > + allow-update { none; }; > + }; > + > + zone "localhost" IN { > + type master; > + file "localhost.zone"; > + allow-update { none; }; > + }; > + > + zone "0.0.127.in-addr.arpa" IN { > + type master; > + file "named.local"; > + allow-update { none; }; > + }; > + > + zone "1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.ip6.arpa" IN { > + type master; > + file "named.ip6.local"; > + allow-update { none; }; > + }; > + > + zone "255.in-addr.arpa" IN { > + type master; > + file "named.broadcast"; > + allow-update { none; }; > + }; > + > + zone "0.in-addr.arpa" IN { > + type master; > + file "named.zero"; > + allow-update { none; }; > + }; > + > +Create /var/named/localdomain.zone as: > + $TTL 86400 > + @ IN SOA localhost root ( > + 42 ; serial (d. adams) > + 3H ; refresh > + 15M ; retry > + 1W ; expiry > + 1D ) ; minimum > + IN NS localhost > + localhost IN A 127.0.0.1 > + > +Create /var/named/localhost.zone > + $TTL 86400 > + @ IN SOA @ root ( > + 42 ; serial (d. adams) > + 3H ; refresh > + 15M ; retry > + 1W ; expiry > + 1D ) ; minimum > + > + IN NS @ > + IN A 127.0.0.1 > + IN AAAA ::1 > + > +Create /var/named/named.local as: > + $TTL 86400 > + @ IN SOA localhost. root.localhost. ( > + 1997022700 ; Serial > + 28800 ; Refresh > + 14400 ; Retry > + 3600000 ; Expire > + 86400 ) ; Minimum > + IN NS localhost. > + 1 IN PTR localhost. > + > +Create /var/named/named.ip6.local as: > + $TTL 86400 > + @ IN SOA localhost. root.localhost. ( > + 1997022700 ; Serial > + 28800 ; Refresh > + 14400 ; Retry > + 3600000 ; Expire > + 86400 ) ; Minimum > + IN NS localhost. > + 1 IN PTR localhost. > + > +Create /var/named/named.broadcast as: > + $TTL 86400 > + @ IN SOA localhost. root.localhost. ( > + 42 ; serial (d. adams) > + 3H ; refresh > + 15M ; retry > + 1W ; expiry > + 1D ) ; minimum > + IN NS localhost. > + > +Create /var/named/named.zero as: > + $TTL 86400 > + @ IN SOA localhost. root.localhost. ( > + 42 ; serial (d. adams) > + 3H ; refresh > + 15M ; retry > + 1W ; expiry > + 1D ) ; minimum > + IN NS localhost. > + > +Create /etc/named.conf as: > + options { > + listen-on port 53 { any; }; > + listen-on-v6 port 53 { ::1; }; > + directory "/var/named"; > + allow-query { any; }; > + forwarders { 192.168.1.1; }; > + recursion yes; > + }; > + > + logging { > + channel default_debug { > + file "data/named.run"; > + severity dynamic; > + }; > + }; > + > + zone "." IN { > + type hint; > + file "named.root"; > + }; > + > + include "/etc/named.rfc1912.zones"; > + > + include "/var/named/master/fedfs.org/zone.fedfs.org"; > + > +Create /var/named/master/fedfs.org/zone.fedfs.org as: > + zone "fedfs.org" { > + type master; > + notify no; > + file "master/fedfs.org/db.fedfs.org"; > + }; > + > + zone "1.168.192.in-addr.arpa" { > + type master; > + notify no; > + file "master/fedfs.org/db.192.168.1"; > + }; > + > +Create /var/named/master/fedfs.org/db.fedfs.org as: > + $TTL 3D > + @ IN SOA ns.fedfs.org. hostmaster.fedfs.org. ( > + 199802152 ; serial, todays date + todays serial # > + 8H ; refresh, seconds > + 2H ; retry, seconds > + 4W ; expire, seconds > + 1D ) ; minimum, seconds > + ; > + NS ns ; Inet Address of name server > + ; > + localhost A 127.0.0.1 > + ns A 192.168.1.32 > + zeus A 192.168.1.32 > + perseus A 192.168.1.31 > + > + ; Assume these aren't present, they've been left to show an example > + ; of the SRV record that would be used for a read-write FedFS domain > + ; rather than the read-only setup we use here. > + ;_nfs4._domainroot._tcp SRV 0 0 2049 perseus.fedfs.org. > + ;_nfs4._write._domainroot._tcp SRV 0 0 2049 perseus.fedfs.org. > + > +Create /var/named/master/fedfs.org/db.192.168.1 as: > + $TTL 3D > + @ IN SOA ns.fedfs.org. hostmaster.fedfs.org. ( > + 199802151 ; Serial, todays date + todays serial > + 8H ; Refresh > + 2H ; Retry > + 4W ; Expire > + 1D) ; Minimum TTL > + NS ns.fedfs.org. > + > + 32 PTR ns.fedfs.org. > + 32 PTR zeus.fedfs.org. > + 31 PTR perseus.fedfs.org. > + > + service named start > +or > + systemctl start named.service > + > +2. On perseus.fedfs.org and zeus.fedfs.org configure /etc/resolve.conf: > + cd /etc > + cp resolve.conf resolv.conf.orig > + echo "domain fedfs.org" > resolve.conf > + echo "search fedfs.org" >> resolve.conf > + echo "nameserver 192.168.1.32" >> resolve.conf > + > + > +Setup an NSDB (NameSpace DataBase) > +================================== > + > +1. Set parameters for NSDB connections: > + # nsdbparams(8) is used to set NSDB connection parameters > + nsdbparams update -e "dc=fedfs,dc=org" \ > + -D "cn=Manager,dc=fedfs,dc=org" \ > + zeus.fedfs.org Note: This command is run on file servers and NSDB administrative clients, not on the LDAP server, so those servers can contact the LDAP server correctly. Here's where I'm a little confused about whether this is perseus or zeus. > + > +2. Setup an OpenLDAP instance for fedfs.org: > + service slapd stop > +or > + systemctl stop slapd.service > + > +Create /etc/openldap/slapd.conf as: > + include /etc/openldap/schema/core.schema > + include /etc/openldap/schema/cosine.schema > + include /etc/openldap/schema/inetorgperson.schema > + include /etc/openldap/schema/nis.schema > + # Get this from the FedFS distribution > + include /etc/openldap/schema/fedfs.schema > + > + pidfile /var/run/openldap/slapd.pid > + argsfile /var/run/openldap/slapd.args > + > + database bdb > + suffix "dc=fedfs,dc=org" > + rootdn "cn=Manager,dc=fedfs,dc=org" > + rootpw secret > + directory /var/lib/ldap > + > + index objectClass eq,pres > + index ou,cn,mail,surname,givenname eq,pres,sub > + index uidNumber,gidNumber,loginShell eq,pres > + index uid,memberUid eq,pres,sub > + index nisMapName,nisMapEntry eq,pres,sub > + > +Create fedfs.org-naming-context.ldif as: > + dn: dc=fedfs,dc=org > + objectClass: domain > + dc: fedfs > + description: naming context > + > +Create an OpenLDAP instance for fedfs.org: > + # Ensure that /etc/openldap/ldap.conf is setup to use fedfs.org > + # by commenting out any other URI and BASE directives and then > + # add definitions to the end of the configuration. > + echo "URI ldap://zeus.fedfs.org/" >> /etc/openldap/ldap.conf > + echo "BASE dc=fedfs,dc=org" >> /etc/openldap/ldap.conf > + > + # Setup OpenLDAP for fedfs.org > + cd /etc/openldap > + restorecon -v slapd.conf > + > + rm -f /var/lib/ldap/* > + # supress warnings that this file does not exist > + touch /var/lib/ldap/DB_CONFIG > + > + rm -rf slapd.d > + slaptest -F slapd.d -f slapd.conf > + # I think this will restore the correct ownership > + # of the configuration directory tree but it may > + # be necessary to chmod -R ldap.ldap slapd.d also. > + restorecon -R -v slapd.d > + > + slapadd -l fedfs.org-naming-context.ldif > + chown ldap.ldap /var/lib/ldap/* > + restorecon -v /var/lib/ldap/* > + > + service slapd start > +or > + systemctl start slapd.service Is all of this necessary to include here? I'm a detail-oriented person, but I'm little concerned about how long these instructions are (for DNS and LDAP set-up), since it gives the impression of a lot of complexity (yes, I know, there are already a lot of moving parts here). But one would hope that installing and configuring an LDAP server would be simpler… > + > +4. Add NCI (NSDB Container information) attributes to the > + naming context LDAP entry: > + nsdb-update-nci -l zeus.fedfs.org \ > + -D "cn=Manager,dc=fedfs,dc=org" \ > + -e "dc=fedfs,dc=org" > + > + > +Add a FedFS junction within a domain root directory > +=================================================== > + > +Assuming there is a file system mounted on /vm (or just a directory > +we can export) on server zues.fedfs.org which we want to access > +under the domain root as <domain root mount point>/vm, we will be > +exporting /.domainroot-fedfs.org as the domain root. > + > +1. Add an entry to /etc/exports on zeus.fedfs.org: > + > + # Add to /etc/exports > + /vm *(ro) > + > + # Restart the nfs service or just re-export the table > + exportfs -r Why not just create and export a directory called "/.domainroot-fedfs.org" here? Alternatively you could describe how to use a bind mount to export some other FS as "/.domainroot-fedfs.org." > + > +2. Add a junction to the domain root on persues.fedfs.org: Actually, the domain root is the top of the namespace tree, so you should create junctions in the domain root, not point junctions to it. > + > + # > + # Tell nfsref the LDAP server (the NSDB) we are using to > + # record file system name (FSN) and file system location > + # (FSL) uuids. This assumes the LDAP connection parameters > + # have been setup as in step 1 of "Setup an NSDB". > + # > + export FEDFS_NSDB_HOST=zeus.fedfs.org > + > + # > + # Add the junction metadata to the directory and update > + # the NSDB with uuid info of the junction. > + # > + mkdir -p /.domainroot-fedfs.org/vm > + nfsref --type=nfs-fedfs \ > + add /.domainroot-fedfs.org/vm \ > + zeus.fedfs.org /vm > + > + > +Setup fedfs domain root export (read-only case) > +=============================================== > + > +For this we are seeking to mount the domain root exported from host > +perseus.fedfs.org. > + > +1. Add an SRV record for the FedFS file server to DNS: > + > + _nfs4._domainroot._tcp SRV 0 0 2049 perseus.fedfs.org. > + > +2. Restart named to make in available. > + > + service named restart > +or > + systemctl restart named.service Or "rndc reload" > + > +3. Add an entry to /etc/exports on perseus.autofs.test: > + > + # > + # Created when we added the junction above. > + # mkdir /.domainroot-fedfs.org > + # > + /.domainroot-fedfs.org *(ro) > + > +4. Restart NFS: > + > + service nfs restart > +or > + systemctl restart nfs.service Or "exportfs -rv" > + > +5. Mount using the FedFS mount utility on a local directory: > + > + mount -v -t fedfs /nfs4/fedfs.org /mnt > + mount | grep domainroot > + perseus.fedfs.org:/.domainroot-fedfs.org/ on /mnt type nfs4 ... > + > + cd /mnt/vm > + > + # > + # This check assumes /etc/mtab is symlinked to /proc/mounts > + # as it is in Fedora. Kernel automounted file systems will > + # not be present in the text based /etc/mtab and so will not > + # be seen in it. Look to /proc/mounts instead in this case. > + # > + mount | grep ^zeus.fedfs.org.*vm > + zeus.fedfs.org:/vm/ on /mnt/vm type nfs4 ... > + > + # Ha, move out of the directory so it can be umounted > + cd > + > +5. Lastly cleanup: 6. :-) > + > + # > + # This example includes a specific umount of the junction > + # (/mnt/vm) but such kernel automounted file systems are > + # umounted automatically (when they are not in use) so it > + # may not be present when this step is done. > + # > + umount /mnt/vm > + umount /mnt > + > +Setup autofs to automount the domain root > +========================================= > + > +1. Add a line to /etc/auto.master to automount FedFS root domains: > + > + # > + # Note that the autofs pseudo option "nobind" probably > + # should be used. In the case here it is required because > + # the FedFS client also hosts the root of the domain and > + # autofs will see the mount is local and perform a bind > + # mount instead of an NFS mount. That, of course, means > + # file system lookups won't be with an NFS file system > + # so NFS referals can't be followed. > + # > + # Also note that the autofs mount point name must be > + # /nfs4 to be able to mount nfs4 root domains. > + # > + echo "/nfs4 /usr/sbin/fedfs-map-nfs4 nobind" >> /etc/auto.master > + > +2. Restart or reload the autofs service: > + > + service autofs restart > +or > + systemctl restart autofs.service > + > +3. Check that we can mount the domain root and the referal: > + > + # automount the root domain. > + [raven@perseus ~]$ ls /nfs4/fedfs.org > + top.txt vm > + > + # automount the referal (from a different machine). > + [raven@perseus ~]$ ls /nfs4/fedfs.org/vm > + lost+found test.txt > + > + # Check they were mounted. > + [raven@perseus ~]$ mount |grep perseus|grep nfs4 > + perseus.fedfs.org:/.domainroot-fedfs.org/ on /nfs4/fedfs.org type nfs4 ... > + [raven@perseus ~]$ mount |grep zeus|grep nfs4 > + zeus.fedfs.org:/vm/ on /nfs4/fedfs.org/vm type nfs4 … Or "findmnt -t nfs4" -- Chuck Lever chuck[dot]lever[at]oracle[dot]com
On Mon, 2012-08-27 at 08:24 -0700, Chuck Lever wrote: > Hi Ian- > > FYI, we've got a patchwork repository, your patch is now here: > > http://patchwork.ozlabs.org/patch/180159/ > > And I marked it "Under Review." Thanks for tackling this. Overall, > the organization is right, but the details need a little work, and > hopefully I've given you some pointers in the right direction. > > Also there is an INSTALL file in the tar ball. Should we augment that > instead? Not sure. I think we'll have to wait and see what we end up with before deciding that. The scope is a bit to broad to be included in install instructions at this stage. > > > On Aug 27, 2012, at 2:28 AM, Ian Kent wrote: > > > Add a howto to setup a DNS domain fedfs.org, an (OpenLDAP) LDAP > > instance with context dc=fedfs,dc=org and work through the steps > > to setup a read-only FedFS root domain with a single referral. > > > > Signed-off-by: Ian Kent <raven@themaw.net> > > --- > > > > doc/howto/fedfs-simple-setup-howto.txt | 465 ++++++++++++++++++++++++++++++++ > > 1 files changed, 465 insertions(+), 0 deletions(-) > > create mode 100644 doc/howto/fedfs-simple-setup-howto.txt > > > > diff --git a/doc/howto/fedfs-simple-setup-howto.txt b/doc/howto/fedfs-simple-setup-howto.txt > > new file mode 100644 > > index 0000000..6454f64 > > --- /dev/null > > +++ b/doc/howto/fedfs-simple-setup-howto.txt > > @@ -0,0 +1,465 @@ > > + > > +Simple example of basic FedFS setup and operation > > +================================================= > > + > > +The examples in this test procedure assume a Fedora 18 install. > > + > > +Assume that the package fedfs-utils-server and fedfs-utils-client > > +are installed on perseus.fedfs.org. And that fedfs-utils-server and > > +a nfs-utils built with fedfs-utils-devel installed on zeus.fedfs.org > > +and that fedfs-utils-lib is also installed on zeus.fedfs.org. > > Since fedfs-utils is nominally the "upstream" version, we should > discuss the tar ball installation procedure here, and not a particular > distributor's package installation. OK, so that just means saying something like fedfs-utils has been build and installed on each machine, right. > > > + > > +We want to setup perseus.fedfs.org to provide the domain root, > > +be able to use FedFS to mount the domain root, and to setup and > > +use a FedFS juntion to access an export on zeus.fedfs.org upon > > +access to a directory within the domain root. And lastly, setup > > +autofs to use the FedFS program map to mount the root of the > > +domain. > > There are five roles in a FedFS domain: file server, filesystem client, > NSDB, NSDB admin client, and DNS server. I find that helps organize > the set-up steps. > > It's not always clear below where you are switching between perseus and > zeus, nor which host is running the forwarding DNS or the LDAP > service. It would be good to clarify that. But also I think at least > the filesystem client (which mounts the domain root or which runs > autofs) ought to be a separate host. That did get a bit lost in writing this. The idea was to use only two machines, one for LDAP, DNS and NSDB, and as an NFS server with a single export to be used as a referral (specifically an export on a different machine from the machine exporting the domain root) and a client which was also an NFS server exporting the domain root. This division happened because I had easy access to two machines. I'm not sure that's a bad thing though because using only two machines will make it easier for someone to play with the functionality before starting out on setting up something in a production testing environment. The other goal of the procedure was to outline how to setup a domain root that is stand-alone and independent of an exiting environment so that setup in a production test environment is a straight forward extension of the procedure. The thing that is still missing I think is usage of sub domains (either delegated or included within an existing DNS configuration). But I don't think we have a discovery protocol yet anyway. > > > + > > +#Questions still to be answered: > > +#1. When and where (ie. on which server) is rpc.fedfsd used in the > > +# procedure below. > > rpc.fedfsd is set up on the NFS file servers in the domain, including > the file server that is sharing the domain root. Note, though, that > it's optional: you can do the same operations by running "nfsref" or > "nsdbparams" directly on file servers. I got that but, possibly due to the multiple functions of each of the machines, I think I saw some unexpected contacts to the daemon. I didn't focus on that though so I can't be sure. I'll need to re-run the procedure and check. Probably best to do that when the document is further along. > > > +#2. What is the procedure for setting up a junction on a third > > +# unrelated machine. > > Works pretty much like setting up the domain root. Pick an export on > some file server, and use "nfsref" to set up a junction. Probably worth adding a description of that too. > > > + > > +Setup a forwarding DNS server for fedfs.org > > +=========================================== > > + > > +Some of this setup may not be necessary but it is what I use to setup > > +DNS instances for testsing. Also many of the files created in step 1 > > +should be included in a caching nameserver package. Installing the > > +cacheing nameserver package and making appropriate modifications to > > +turn the cacheing nameserver into a forwarding nameserver iis really > > +all that needs to be done. Of course the fedfs zone files will always > > +be needed. > > + > > +The example assumes the private IP address range 192.168.1 is being > > +used and a nameserver that is able to satisfy name resolution for > > +external names is at 192.168.1.1 and the hosts perseus.fedfs.org and > > +zeus.fedfs.org are assigned addresses 192.168.1.32 and 192.168.1.31 > > +respectively. > > + > > +1. Setup zeus.fedfs.org as a forwarding DNS server: > > Using a forwarding server is an interesting choice… we often do it this > way for testing. But the point of the SRV record is to advertise the > domain root globally. Should we describe here how to add an SRV to a > non-forwarding DNS server instead, or in addition to, the below recipe? > > Ah, much farther along, you set up the SRV record. > > > + > > + mkdir -p /var/named/master/fedfs.org > > + if [ ! -d /var/named/data ]; then > > + mkdir /var/named/data > > + fi > > + > > +Create /var/named/named.root using: > > + dig +norec NS > /var/named/named.root > > + > > +Create /etc/named.rfc1912.zones as: > > + // named.rfc1912.zones: > > + // > > + // ISC BIND named zone configuration for zones recommended by > > + // RFC 1912 section 4.1 : localhost TLDs and address zones > > + // > > + zone "localdomain" IN { > > + type master; > > + file "localdomain.zone"; > > + allow-update { none; }; > > + }; > > + > > + zone "localhost" IN { > > + type master; > > + file "localhost.zone"; > > + allow-update { none; }; > > + }; > > + > > + zone "0.0.127.in-addr.arpa" IN { > > + type master; > > + file "named.local"; > > + allow-update { none; }; > > + }; > > + > > + zone "1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.ip6.arpa" IN { > > + type master; > > + file "named.ip6.local"; > > + allow-update { none; }; > > + }; > > + > > + zone "255.in-addr.arpa" IN { > > + type master; > > + file "named.broadcast"; > > + allow-update { none; }; > > + }; > > + > > + zone "0.in-addr.arpa" IN { > > + type master; > > + file "named.zero"; > > + allow-update { none; }; > > + }; > > + > > +Create /var/named/localdomain.zone as: > > + $TTL 86400 > > + @ IN SOA localhost root ( > > + 42 ; serial (d. adams) > > + 3H ; refresh > > + 15M ; retry > > + 1W ; expiry > > + 1D ) ; minimum > > + IN NS localhost > > + localhost IN A 127.0.0.1 > > + > > +Create /var/named/localhost.zone > > + $TTL 86400 > > + @ IN SOA @ root ( > > + 42 ; serial (d. adams) > > + 3H ; refresh > > + 15M ; retry > > + 1W ; expiry > > + 1D ) ; minimum > > + > > + IN NS @ > > + IN A 127.0.0.1 > > + IN AAAA ::1 > > + > > +Create /var/named/named.local as: > > + $TTL 86400 > > + @ IN SOA localhost. root.localhost. ( > > + 1997022700 ; Serial > > + 28800 ; Refresh > > + 14400 ; Retry > > + 3600000 ; Expire > > + 86400 ) ; Minimum > > + IN NS localhost. > > + 1 IN PTR localhost. > > + > > +Create /var/named/named.ip6.local as: > > + $TTL 86400 > > + @ IN SOA localhost. root.localhost. ( > > + 1997022700 ; Serial > > + 28800 ; Refresh > > + 14400 ; Retry > > + 3600000 ; Expire > > + 86400 ) ; Minimum > > + IN NS localhost. > > + 1 IN PTR localhost. > > + > > +Create /var/named/named.broadcast as: > > + $TTL 86400 > > + @ IN SOA localhost. root.localhost. ( > > + 42 ; serial (d. adams) > > + 3H ; refresh > > + 15M ; retry > > + 1W ; expiry > > + 1D ) ; minimum > > + IN NS localhost. > > + > > +Create /var/named/named.zero as: > > + $TTL 86400 > > + @ IN SOA localhost. root.localhost. ( > > + 42 ; serial (d. adams) > > + 3H ; refresh > > + 15M ; retry > > + 1W ; expiry > > + 1D ) ; minimum > > + IN NS localhost. > > + > > +Create /etc/named.conf as: > > + options { > > + listen-on port 53 { any; }; > > + listen-on-v6 port 53 { ::1; }; > > + directory "/var/named"; > > + allow-query { any; }; > > + forwarders { 192.168.1.1; }; > > + recursion yes; > > + }; > > + > > + logging { > > + channel default_debug { > > + file "data/named.run"; > > + severity dynamic; > > + }; > > + }; > > + > > + zone "." IN { > > + type hint; > > + file "named.root"; > > + }; > > + > > + include "/etc/named.rfc1912.zones"; > > + > > + include "/var/named/master/fedfs.org/zone.fedfs.org"; > > + > > +Create /var/named/master/fedfs.org/zone.fedfs.org as: > > + zone "fedfs.org" { > > + type master; > > + notify no; > > + file "master/fedfs.org/db.fedfs.org"; > > + }; > > + > > + zone "1.168.192.in-addr.arpa" { > > + type master; > > + notify no; > > + file "master/fedfs.org/db.192.168.1"; > > + }; > > + > > +Create /var/named/master/fedfs.org/db.fedfs.org as: > > + $TTL 3D > > + @ IN SOA ns.fedfs.org. hostmaster.fedfs.org. ( > > + 199802152 ; serial, todays date + todays serial # > > + 8H ; refresh, seconds > > + 2H ; retry, seconds > > + 4W ; expire, seconds > > + 1D ) ; minimum, seconds > > + ; > > + NS ns ; Inet Address of name server > > + ; > > + localhost A 127.0.0.1 > > + ns A 192.168.1.32 > > + zeus A 192.168.1.32 > > + perseus A 192.168.1.31 > > + > > + ; Assume these aren't present, they've been left to show an example > > + ; of the SRV record that would be used for a read-write FedFS domain > > + ; rather than the read-only setup we use here. > > + ;_nfs4._domainroot._tcp SRV 0 0 2049 perseus.fedfs.org. > > + ;_nfs4._write._domainroot._tcp SRV 0 0 2049 perseus.fedfs.org. > > + > > +Create /var/named/master/fedfs.org/db.192.168.1 as: > > + $TTL 3D > > + @ IN SOA ns.fedfs.org. hostmaster.fedfs.org. ( > > + 199802151 ; Serial, todays date + todays serial > > + 8H ; Refresh > > + 2H ; Retry > > + 4W ; Expire > > + 1D) ; Minimum TTL > > + NS ns.fedfs.org. > > + > > + 32 PTR ns.fedfs.org. > > + 32 PTR zeus.fedfs.org. > > + 31 PTR perseus.fedfs.org. > > + > > + service named start > > +or > > + systemctl start named.service > > + > > +2. On perseus.fedfs.org and zeus.fedfs.org configure /etc/resolve.conf: > > + cd /etc > > + cp resolve.conf resolv.conf.orig > > + echo "domain fedfs.org" > resolve.conf > > + echo "search fedfs.org" >> resolve.conf > > + echo "nameserver 192.168.1.32" >> resolve.conf > > + > > + > > +Setup an NSDB (NameSpace DataBase) > > +================================== > > + > > +1. Set parameters for NSDB connections: > > + # nsdbparams(8) is used to set NSDB connection parameters > > + nsdbparams update -e "dc=fedfs,dc=org" \ > > + -D "cn=Manager,dc=fedfs,dc=org" \ > > + zeus.fedfs.org > > Note: This command is run on file servers and NSDB administrative > clients, not on the LDAP server, so those servers can contact the LDAP > server correctly. Here's where I'm a little confused about whether > this is perseus or zeus. Right, that needs clarification. > > > + > > +2. Setup an OpenLDAP instance for fedfs.org: > > + service slapd stop > > +or > > + systemctl stop slapd.service > > + > > +Create /etc/openldap/slapd.conf as: > > + include /etc/openldap/schema/core.schema > > + include /etc/openldap/schema/cosine.schema > > + include /etc/openldap/schema/inetorgperson.schema > > + include /etc/openldap/schema/nis.schema > > + # Get this from the FedFS distribution > > + include /etc/openldap/schema/fedfs.schema > > + > > + pidfile /var/run/openldap/slapd.pid > > + argsfile /var/run/openldap/slapd.args > > + > > + database bdb > > + suffix "dc=fedfs,dc=org" > > + rootdn "cn=Manager,dc=fedfs,dc=org" > > + rootpw secret > > + directory /var/lib/ldap > > + > > + index objectClass eq,pres > > + index ou,cn,mail,surname,givenname eq,pres,sub > > + index uidNumber,gidNumber,loginShell eq,pres > > + index uid,memberUid eq,pres,sub > > + index nisMapName,nisMapEntry eq,pres,sub > > + > > +Create fedfs.org-naming-context.ldif as: > > + dn: dc=fedfs,dc=org > > + objectClass: domain > > + dc: fedfs > > + description: naming context > > + > > +Create an OpenLDAP instance for fedfs.org: > > + # Ensure that /etc/openldap/ldap.conf is setup to use fedfs.org > > + # by commenting out any other URI and BASE directives and then > > + # add definitions to the end of the configuration. > > + echo "URI ldap://zeus.fedfs.org/" >> /etc/openldap/ldap.conf > > + echo "BASE dc=fedfs,dc=org" >> /etc/openldap/ldap.conf > > + > > + # Setup OpenLDAP for fedfs.org > > + cd /etc/openldap > > + restorecon -v slapd.conf > > + > > + rm -f /var/lib/ldap/* > > + # supress warnings that this file does not exist > > + touch /var/lib/ldap/DB_CONFIG > > + > > + rm -rf slapd.d > > + slaptest -F slapd.d -f slapd.conf > > + # I think this will restore the correct ownership > > + # of the configuration directory tree but it may > > + # be necessary to chmod -R ldap.ldap slapd.d also. > > + restorecon -R -v slapd.d > > + > > + slapadd -l fedfs.org-naming-context.ldif > > + chown ldap.ldap /var/lib/ldap/* > > + restorecon -v /var/lib/ldap/* > > + > > + service slapd start > > +or > > + systemctl start slapd.service > > Is all of this necessary to include here? I'm a detail-oriented > person, but I'm little concerned about how long these instructions are > (for DNS and LDAP set-up), since it gives the impression of a lot of > complexity (yes, I know, there are already a lot of moving parts > here). But one would hope that installing and configuring an LDAP > server would be simpler… That was the dilemma when I set out to do this. The aim of having a procedure to setup something completely standalone means that we need this stuff somewhere but it completely obscures the actual message of FedFS usage. The DNS configuration is fairly standard and much of the configuration would be present in any caching nameserver install so not a lot more is needed. But LDAP has proved to be rather painful over time and there are elements in the procedure that are useful for demonstrating those gotchas so it's worth keeping somewhere. For example, the OpenLDAP change to the directory based setup under slapd.d is problematic when setting up a test instance. Even though removing slapd.d and just using slapd.conf will work (still I believe) it probably isn't a good idea to do it that way since the ability to use slapd.conf might be removed at some point. Another issue is that if the suffix (the dc=fedfs,dc=org) is changed and a default installed database exists then, AFAICT, the LDAP database must be deleted and re-created. Not only that, it appears that the directory based configuration can't be properly updated in this case and must be re-created, but I probably just don't know how to do it. Finally, at least on Fedora, the ownership (and selinux context settings) must be correct for the LDAP server to start. So I think we need to capture this somewhere but probably not right here. > > > + > > +4. Add NCI (NSDB Container information) attributes to the > > + naming context LDAP entry: > > + nsdb-update-nci -l zeus.fedfs.org \ > > + -D "cn=Manager,dc=fedfs,dc=org" \ > > + -e "dc=fedfs,dc=org" > > + > > + > > +Add a FedFS junction within a domain root directory > > +=================================================== > > + > > +Assuming there is a file system mounted on /vm (or just a directory > > +we can export) on server zues.fedfs.org which we want to access > > +under the domain root as <domain root mount point>/vm, we will be > > +exporting /.domainroot-fedfs.org as the domain root. > > + > > +1. Add an entry to /etc/exports on zeus.fedfs.org: > > + > > + # Add to /etc/exports > > + /vm *(ro) > > + > > + # Restart the nfs service or just re-export the table > > + exportfs -r > > Why not just create and export a directory called > "/.domainroot-fedfs.org" here? Alternatively you could describe how to > use a bind mount to export some other FS as "/.domainroot-fedfs.org." This is out of order. The exporting of the root domain on perseus below should have come before this. And using only two machines lends to confusion mmm .... > > > + > > +2. Add a junction to the domain root on persues.fedfs.org: > > Actually, the domain root is the top of the namespace tree, so you > should create junctions in the domain root, not point junctions to it. Right but I think the aim of having the referral on a different machine and the out of order procedure has obscured this. > > > + > > + # > > + # Tell nfsref the LDAP server (the NSDB) we are using to > > + # record file system name (FSN) and file system location > > + # (FSL) uuids. This assumes the LDAP connection parameters > > + # have been setup as in step 1 of "Setup an NSDB". > > + # > > + export FEDFS_NSDB_HOST=zeus.fedfs.org > > + > > + # > > + # Add the junction metadata to the directory and update > > + # the NSDB with uuid info of the junction. > > + # > > + mkdir -p /.domainroot-fedfs.org/vm > > + nfsref --type=nfs-fedfs \ > > + add /.domainroot-fedfs.org/vm \ > > + zeus.fedfs.org /vm > > + > > + > > +Setup fedfs domain root export (read-only case) > > +=============================================== > > + > > +For this we are seeking to mount the domain root exported from host > > +perseus.fedfs.org. > > + > > +1. Add an SRV record for the FedFS file server to DNS: > > + > > + _nfs4._domainroot._tcp SRV 0 0 2049 perseus.fedfs.org. > > + > > +2. Restart named to make in available. > > + > > + service named restart > > +or > > + systemctl restart named.service > > Or "rndc reload" Not sure, it's been a long time since I maintained DNS servers so I can't remember. I thought there was an extra bit of configuration for this to work properly .... > > > + > > +3. Add an entry to /etc/exports on perseus.autofs.test: > > + > > + # > > + # Created when we added the junction above. > > + # mkdir /.domainroot-fedfs.org > > + # > > + /.domainroot-fedfs.org *(ro) > > + > > +4. Restart NFS: > > + > > + service nfs restart > > +or > > + systemctl restart nfs.service > > Or "exportfs -rv" Yeah, my habits get in the road, ;) > > > + > > +5. Mount using the FedFS mount utility on a local directory: > > + > > + mount -v -t fedfs /nfs4/fedfs.org /mnt > > + mount | grep domainroot > > + perseus.fedfs.org:/.domainroot-fedfs.org/ on /mnt type nfs4 ... > > + > > + cd /mnt/vm > > + > > + # > > + # This check assumes /etc/mtab is symlinked to /proc/mounts > > + # as it is in Fedora. Kernel automounted file systems will > > + # not be present in the text based /etc/mtab and so will not > > + # be seen in it. Look to /proc/mounts instead in this case. > > + # > > + mount | grep ^zeus.fedfs.org.*vm > > + zeus.fedfs.org:/vm/ on /mnt/vm type nfs4 ... > > + > > + # Ha, move out of the directory so it can be umounted > > + cd > > + > > +5. Lastly cleanup: > > 6. :-) Ha, ;) > > > + > > + # > > + # This example includes a specific umount of the junction > > + # (/mnt/vm) but such kernel automounted file systems are > > + # umounted automatically (when they are not in use) so it > > + # may not be present when this step is done. > > + # > > + umount /mnt/vm > > + umount /mnt > > + > > +Setup autofs to automount the domain root > > +========================================= > > + > > +1. Add a line to /etc/auto.master to automount FedFS root domains: > > + > > + # > > + # Note that the autofs pseudo option "nobind" probably > > + # should be used. In the case here it is required because > > + # the FedFS client also hosts the root of the domain and > > + # autofs will see the mount is local and perform a bind > > + # mount instead of an NFS mount. That, of course, means > > + # file system lookups won't be with an NFS file system > > + # so NFS referals can't be followed. > > + # > > + # Also note that the autofs mount point name must be > > + # /nfs4 to be able to mount nfs4 root domains. > > + # > > + echo "/nfs4 /usr/sbin/fedfs-map-nfs4 nobind" >> /etc/auto.master > > + > > +2. Restart or reload the autofs service: > > + > > + service autofs restart > > +or > > + systemctl restart autofs.service > > + > > +3. Check that we can mount the domain root and the referal: > > + > > + # automount the root domain. > > + [raven@perseus ~]$ ls /nfs4/fedfs.org > > + top.txt vm > > + > > + # automount the referal (from a different machine). > > + [raven@perseus ~]$ ls /nfs4/fedfs.org/vm > > + lost+found test.txt > > + > > + # Check they were mounted. > > + [raven@perseus ~]$ mount |grep perseus|grep nfs4 > > + perseus.fedfs.org:/.domainroot-fedfs.org/ on /nfs4/fedfs.org type nfs4 ... > > + [raven@perseus ~]$ mount |grep zeus|grep nfs4 > > + zeus.fedfs.org:/vm/ on /nfs4/fedfs.org/vm type nfs4 … > > Or "findmnt -t nfs4" Again, my habits ... > > > -- > Chuck Lever > chuck[dot]lever[at]oracle[dot]com > > > > > _______________________________________________ > fedfs-utils-devel mailing list > fedfs-utils-devel@oss.oracle.com > https://oss.oracle.com/mailman/listinfo/fedfs-utils-devel
On Aug 28, 2012, at 8:45 PM, Ian Kent wrote: > On Mon, 2012-08-27 at 08:24 -0700, Chuck Lever wrote: >> Hi Ian- >> >> FYI, we've got a patchwork repository, your patch is now here: >> >> http://patchwork.ozlabs.org/patch/180159/ >> >> And I marked it "Under Review." Thanks for tackling this. Overall, >> the organization is right, but the details need a little work, and >> hopefully I've given you some pointers in the right direction. >> >> Also there is an INSTALL file in the tar ball. Should we augment that >> instead? > > Not sure. I think we'll have to wait and see what we end up with before > deciding that. The scope is a bit to broad to be included in install > instructions at this stage. > >> >> >> On Aug 27, 2012, at 2:28 AM, Ian Kent wrote: >> >>> Add a howto to setup a DNS domain fedfs.org, an (OpenLDAP) LDAP >>> instance with context dc=fedfs,dc=org and work through the steps >>> to setup a read-only FedFS root domain with a single referral. >>> >>> Signed-off-by: Ian Kent <raven@themaw.net> >>> --- >>> >>> doc/howto/fedfs-simple-setup-howto.txt | 465 ++++++++++++++++++++++++++++++++ >>> 1 files changed, 465 insertions(+), 0 deletions(-) >>> create mode 100644 doc/howto/fedfs-simple-setup-howto.txt >>> >>> diff --git a/doc/howto/fedfs-simple-setup-howto.txt b/doc/howto/fedfs-simple-setup-howto.txt >>> new file mode 100644 >>> index 0000000..6454f64 >>> --- /dev/null >>> +++ b/doc/howto/fedfs-simple-setup-howto.txt >>> @@ -0,0 +1,465 @@ >>> + >>> +Simple example of basic FedFS setup and operation >>> +================================================= >>> + >>> +The examples in this test procedure assume a Fedora 18 install. >>> + >>> +Assume that the package fedfs-utils-server and fedfs-utils-client >>> +are installed on perseus.fedfs.org. And that fedfs-utils-server and >>> +a nfs-utils built with fedfs-utils-devel installed on zeus.fedfs.org >>> +and that fedfs-utils-lib is also installed on zeus.fedfs.org. >> >> Since fedfs-utils is nominally the "upstream" version, we should >> discuss the tar ball installation procedure here, and not a particular >> distributor's package installation. > > OK, so that just means saying something like fedfs-utils has been build > and installed on each machine, right. > >> >>> + >>> +We want to setup perseus.fedfs.org to provide the domain root, >>> +be able to use FedFS to mount the domain root, and to setup and >>> +use a FedFS juntion to access an export on zeus.fedfs.org upon >>> +access to a directory within the domain root. And lastly, setup >>> +autofs to use the FedFS program map to mount the root of the >>> +domain. >> >> There are five roles in a FedFS domain: file server, filesystem client, >> NSDB, NSDB admin client, and DNS server. I find that helps organize >> the set-up steps. >> >> It's not always clear below where you are switching between perseus and >> zeus, nor which host is running the forwarding DNS or the LDAP >> service. It would be good to clarify that. But also I think at least >> the filesystem client (which mounts the domain root or which runs >> autofs) ought to be a separate host. > > That did get a bit lost in writing this. > > The idea was to use only two machines, one for LDAP, DNS and NSDB, and > as an NFS server with a single export to be used as a referral > (specifically an export on a different machine from the machine > exporting the domain root) and a client which was also an NFS server > exporting the domain root. > > This division happened because I had easy access to two machines. > I'm not sure that's a bad thing though because using only two machines > will make it easier for someone to play with the functionality before > starting out on setting up something in a production testing > environment. > > The other goal of the procedure was to outline how to setup a domain > root that is stand-alone and independent of an exiting environment so > that setup in a production test environment is a straight forward > extension of the procedure. If you stick with two machines, it might help if the domain root was exported from the DNS/NSDB system, and the second export was on the client. Or, something incremental, like: 1. Set up the /.domainroot export and create some basic junctions 2. Make it discoverable by adding the DNS SRV record and setting up the client's auto mounter program map 3. Set up the NSDB and create real FedFS junctions in the domain root Each step gives more flexibility, but there is something useful going on after each step is complete. > The thing that is still missing I think is usage of sub domains (either > delegated or included within an existing DNS configuration). But I don't > think we have a discovery protocol yet anyway. If I understand correctly, I don't think FedFS has "subdomains." Junctions can appear anywhere in the domain's namespace, in any export. They can even refer to other domains. But as far as I know there isn't the concept of a hierarchy of domains. The top level directory of /nfs4 is a flat namespace. Let me know if I misunderstood. [ … snip … ] > >> >>> + >>> +2. Setup an OpenLDAP instance for fedfs.org: >>> + service slapd stop >>> +or >>> + systemctl stop slapd.service >>> + >>> +Create /etc/openldap/slapd.conf as: >>> + include /etc/openldap/schema/core.schema >>> + include /etc/openldap/schema/cosine.schema >>> + include /etc/openldap/schema/inetorgperson.schema >>> + include /etc/openldap/schema/nis.schema >>> + # Get this from the FedFS distribution >>> + include /etc/openldap/schema/fedfs.schema >>> + >>> + pidfile /var/run/openldap/slapd.pid >>> + argsfile /var/run/openldap/slapd.args >>> + >>> + database bdb >>> + suffix "dc=fedfs,dc=org" >>> + rootdn "cn=Manager,dc=fedfs,dc=org" >>> + rootpw secret >>> + directory /var/lib/ldap >>> + >>> + index objectClass eq,pres >>> + index ou,cn,mail,surname,givenname eq,pres,sub >>> + index uidNumber,gidNumber,loginShell eq,pres >>> + index uid,memberUid eq,pres,sub >>> + index nisMapName,nisMapEntry eq,pres,sub >>> + >>> +Create fedfs.org-naming-context.ldif as: >>> + dn: dc=fedfs,dc=org >>> + objectClass: domain >>> + dc: fedfs >>> + description: naming context >>> + >>> +Create an OpenLDAP instance for fedfs.org: >>> + # Ensure that /etc/openldap/ldap.conf is setup to use fedfs.org >>> + # by commenting out any other URI and BASE directives and then >>> + # add definitions to the end of the configuration. >>> + echo "URI ldap://zeus.fedfs.org/" >> /etc/openldap/ldap.conf >>> + echo "BASE dc=fedfs,dc=org" >> /etc/openldap/ldap.conf >>> + >>> + # Setup OpenLDAP for fedfs.org >>> + cd /etc/openldap >>> + restorecon -v slapd.conf >>> + >>> + rm -f /var/lib/ldap/* >>> + # supress warnings that this file does not exist >>> + touch /var/lib/ldap/DB_CONFIG >>> + >>> + rm -rf slapd.d >>> + slaptest -F slapd.d -f slapd.conf >>> + # I think this will restore the correct ownership >>> + # of the configuration directory tree but it may >>> + # be necessary to chmod -R ldap.ldap slapd.d also. >>> + restorecon -R -v slapd.d >>> + >>> + slapadd -l fedfs.org-naming-context.ldif >>> + chown ldap.ldap /var/lib/ldap/* >>> + restorecon -v /var/lib/ldap/* >>> + >>> + service slapd start >>> +or >>> + systemctl start slapd.service >> >> Is all of this necessary to include here? I'm a detail-oriented >> person, but I'm little concerned about how long these instructions are >> (for DNS and LDAP set-up), since it gives the impression of a lot of >> complexity (yes, I know, there are already a lot of moving parts >> here). But one would hope that installing and configuring an LDAP >> server would be simpler… > > That was the dilemma when I set out to do this. > > The aim of having a procedure to setup something completely standalone > means that we need this stuff somewhere but it completely obscures the > actual message of FedFS usage. > > The DNS configuration is fairly standard and much of the configuration > would be present in any caching nameserver install so not a lot more is > needed. > > But LDAP has proved to be rather painful over time and there are > elements in the procedure that are useful for demonstrating those > gotchas so it's worth keeping somewhere. This is the crux… LDAP is complicated, and the set up recipes depend explicitly on which LDAP server is in use. That's exactly why I have provided NSDB admin tools specifically for setting up the NCI, but I agree that's not yet enough. Here's where we can use some thought about how to make this a simpler process. For example, if we stick to keeping NSDB records under the LDAP server's default domain controller naming context (dc=example,dc=net), which should be created automatically by the LDAP server's installation procedure, and not use a brand new naming context (o=fedfs) maybe there is less LDAP specific set up required (i.e., we can manage more of the set up with the server's default set up instructions, plus a generic tool to turn it into an NSDB). Btw, your recipe may be missing a step: where is the FedFS schema installed on the LDAP server? I'm trying to make that step unneeded by having the LDAP vendors include the schema in their distribution packaging, but that hasn't happened yet. > For example, the OpenLDAP change to the directory based setup under > slapd.d is problematic when setting up a test instance. Even though > removing slapd.d and just using slapd.conf will work (still I believe) > it probably isn't a good idea to do it that way since the ability to use > slapd.conf might be removed at some point. Another issue is that if the > suffix (the dc=fedfs,dc=org) is changed and a default installed database > exists then, AFAICT, the LDAP database must be deleted and re-created. > Not only that, it appears that the directory based configuration can't > be properly updated in this case and must be re-created, but I probably > just don't know how to do it. Finally, at least on Fedora, the ownership > (and selinux context settings) must be correct for the LDAP server to > start. > > So I think we need to capture this somewhere but probably not right > here. > >> >>> + >>> +4. Add NCI (NSDB Container information) attributes to the >>> + naming context LDAP entry: >>> + nsdb-update-nci -l zeus.fedfs.org \ >>> + -D "cn=Manager,dc=fedfs,dc=org" \ >>> + -e "dc=fedfs,dc=org" >>> + >>> + >>> +Add a FedFS junction within a domain root directory >>> +=================================================== >>> + >>> +Assuming there is a file system mounted on /vm (or just a directory >>> +we can export) on server zues.fedfs.org which we want to access >>> +under the domain root as <domain root mount point>/vm, we will be >>> +exporting /.domainroot-fedfs.org as the domain root. >>> + >>> +1. Add an entry to /etc/exports on zeus.fedfs.org: >>> + >>> + # Add to /etc/exports >>> + /vm *(ro) >>> + >>> + # Restart the nfs service or just re-export the table >>> + exportfs -r >> >> Why not just create and export a directory called >> "/.domainroot-fedfs.org" here? Alternatively you could describe how to >> use a bind mount to export some other FS as "/.domainroot-fedfs.org." > > This is out of order. > > The exporting of the root domain on perseus below should have come > before this. And using only two machines lends to confusion mmm .... > >> >>> + >>> +2. Add a junction to the domain root on persues.fedfs.org: >> >> Actually, the domain root is the top of the namespace tree, so you >> should create junctions in the domain root, not point junctions to it. > > Right but I think the aim of having the referral on a different machine > and the out of order procedure has obscured this. -- Chuck Lever chuck[dot]lever[at]oracle[dot]com
On Wed, 2012-08-29 at 13:53 -0700, Chuck Lever wrote: > On Aug 28, 2012, at 8:45 PM, Ian Kent wrote: > > > On Mon, 2012-08-27 at 08:24 -0700, Chuck Lever wrote: > >> Hi Ian- > >> > >> FYI, we've got a patchwork repository, your patch is now here: > >> > >> http://patchwork.ozlabs.org/patch/180159/ > >> > >> And I marked it "Under Review." Thanks for tackling this. Overall, > >> the organization is right, but the details need a little work, and > >> hopefully I've given you some pointers in the right direction. > >> > >> Also there is an INSTALL file in the tar ball. Should we augment that > >> instead? > > > > Not sure. I think we'll have to wait and see what we end up with before > > deciding that. The scope is a bit to broad to be included in install > > instructions at this stage. > > > >> > >> > >> On Aug 27, 2012, at 2:28 AM, Ian Kent wrote: > >> > >>> Add a howto to setup a DNS domain fedfs.org, an (OpenLDAP) LDAP > >>> instance with context dc=fedfs,dc=org and work through the steps > >>> to setup a read-only FedFS root domain with a single referral. > >>> > >>> Signed-off-by: Ian Kent <raven@themaw.net> > >>> --- > >>> > >>> doc/howto/fedfs-simple-setup-howto.txt | 465 ++++++++++++++++++++++++++++++++ > >>> 1 files changed, 465 insertions(+), 0 deletions(-) > >>> create mode 100644 doc/howto/fedfs-simple-setup-howto.txt > >>> > >>> diff --git a/doc/howto/fedfs-simple-setup-howto.txt b/doc/howto/fedfs-simple-setup-howto.txt > >>> new file mode 100644 > >>> index 0000000..6454f64 > >>> --- /dev/null > >>> +++ b/doc/howto/fedfs-simple-setup-howto.txt > >>> @@ -0,0 +1,465 @@ > >>> + > >>> +Simple example of basic FedFS setup and operation > >>> +================================================= > >>> + > >>> +The examples in this test procedure assume a Fedora 18 install. > >>> + > >>> +Assume that the package fedfs-utils-server and fedfs-utils-client > >>> +are installed on perseus.fedfs.org. And that fedfs-utils-server and > >>> +a nfs-utils built with fedfs-utils-devel installed on zeus.fedfs.org > >>> +and that fedfs-utils-lib is also installed on zeus.fedfs.org. > >> > >> Since fedfs-utils is nominally the "upstream" version, we should > >> discuss the tar ball installation procedure here, and not a particular > >> distributor's package installation. > > > > OK, so that just means saying something like fedfs-utils has been build > > and installed on each machine, right. > > > >> > >>> + > >>> +We want to setup perseus.fedfs.org to provide the domain root, > >>> +be able to use FedFS to mount the domain root, and to setup and > >>> +use a FedFS juntion to access an export on zeus.fedfs.org upon > >>> +access to a directory within the domain root. And lastly, setup > >>> +autofs to use the FedFS program map to mount the root of the > >>> +domain. > >> > >> There are five roles in a FedFS domain: file server, filesystem client, > >> NSDB, NSDB admin client, and DNS server. I find that helps organize > >> the set-up steps. > >> > >> It's not always clear below where you are switching between perseus and > >> zeus, nor which host is running the forwarding DNS or the LDAP > >> service. It would be good to clarify that. But also I think at least > >> the filesystem client (which mounts the domain root or which runs > >> autofs) ought to be a separate host. > > > > That did get a bit lost in writing this. > > > > The idea was to use only two machines, one for LDAP, DNS and NSDB, and > > as an NFS server with a single export to be used as a referral > > (specifically an export on a different machine from the machine > > exporting the domain root) and a client which was also an NFS server > > exporting the domain root. > > > > This division happened because I had easy access to two machines. > > I'm not sure that's a bad thing though because using only two machines > > will make it easier for someone to play with the functionality before > > starting out on setting up something in a production testing > > environment. > > > > The other goal of the procedure was to outline how to setup a domain > > root that is stand-alone and independent of an exiting environment so > > that setup in a production test environment is a straight forward > > extension of the procedure. > > If you stick with two machines, it might help if the domain root was > exported from the DNS/NSDB system, and the second export was on the > client. OK, I'll do that. The other thing that occurs is we could assume there are several names like ldap.fedfs.org, nsdb.fedfs.org etc. and, as long as the name is used when referred to in the procedure, it would be clear what the target is. That should make the description independent of the number of machines used. > > Or, something incremental, like: > > 1. Set up the /.domainroot export and create some basic junctions > > 2. Make it discoverable by adding the DNS SRV record and setting up > the client's auto mounter program map > > 3. Set up the NSDB and create real FedFS junctions in the domain root > > Each step gives more flexibility, but there is something useful going > on after each step is complete. OK. > > > The thing that is still missing I think is usage of sub domains (either > > delegated or included within an existing DNS configuration). But I don't > > think we have a discovery protocol yet anyway. > > If I understand correctly, I don't think FedFS has "subdomains." > Junctions can appear anywhere in the domain's namespace, in any > export. They can even refer to other domains. But as far as I know > there isn't the concept of a hierarchy of domains. The top level > directory of /nfs4 is a flat namespace. I'm thinking of just the top level. There's nothing stopping someone setting up multiple name spaces. For example, assume you have a domain company.com and you've divided your DNS up like: company.com sales.company.com engineering.company.com AFAICS each of these can have a name space accessed via: /nfs4/company.com /nfs4/sales.company.com /nfs4/engineering.company.com and these sub-domains could be actually delegated in the top level domain or just prefixed names within the company.com DNS. I admit it isn't great to have the root domain and sub-domains at the same level but I think this can be done and people with larger name spaces will likely do it. Maybe there's an opportunity to think a bit about this before people start asking for a hierarchic name spaces, although it is out of scope for this task. > > [ … snip … ] > > > > >> > >>> + > >>> +2. Setup an OpenLDAP instance for fedfs.org: > >>> + service slapd stop > >>> +or > >>> + systemctl stop slapd.service > >>> + > >>> +Create /etc/openldap/slapd.conf as: > >>> + include /etc/openldap/schema/core.schema > >>> + include /etc/openldap/schema/cosine.schema > >>> + include /etc/openldap/schema/inetorgperson.schema > >>> + include /etc/openldap/schema/nis.schema > >>> + # Get this from the FedFS distribution > >>> + include /etc/openldap/schema/fedfs.schema > >>> + > >>> + pidfile /var/run/openldap/slapd.pid > >>> + argsfile /var/run/openldap/slapd.args > >>> + > >>> + database bdb > >>> + suffix "dc=fedfs,dc=org" > >>> + rootdn "cn=Manager,dc=fedfs,dc=org" > >>> + rootpw secret > >>> + directory /var/lib/ldap > >>> + > >>> + index objectClass eq,pres > >>> + index ou,cn,mail,surname,givenname eq,pres,sub > >>> + index uidNumber,gidNumber,loginShell eq,pres > >>> + index uid,memberUid eq,pres,sub > >>> + index nisMapName,nisMapEntry eq,pres,sub > >>> + > >>> +Create fedfs.org-naming-context.ldif as: > >>> + dn: dc=fedfs,dc=org > >>> + objectClass: domain > >>> + dc: fedfs > >>> + description: naming context > >>> + > >>> +Create an OpenLDAP instance for fedfs.org: > >>> + # Ensure that /etc/openldap/ldap.conf is setup to use fedfs.org > >>> + # by commenting out any other URI and BASE directives and then > >>> + # add definitions to the end of the configuration. > >>> + echo "URI ldap://zeus.fedfs.org/" >> /etc/openldap/ldap.conf > >>> + echo "BASE dc=fedfs,dc=org" >> /etc/openldap/ldap.conf > >>> + > >>> + # Setup OpenLDAP for fedfs.org > >>> + cd /etc/openldap > >>> + restorecon -v slapd.conf > >>> + > >>> + rm -f /var/lib/ldap/* > >>> + # supress warnings that this file does not exist > >>> + touch /var/lib/ldap/DB_CONFIG > >>> + > >>> + rm -rf slapd.d > >>> + slaptest -F slapd.d -f slapd.conf > >>> + # I think this will restore the correct ownership > >>> + # of the configuration directory tree but it may > >>> + # be necessary to chmod -R ldap.ldap slapd.d also. > >>> + restorecon -R -v slapd.d > >>> + > >>> + slapadd -l fedfs.org-naming-context.ldif > >>> + chown ldap.ldap /var/lib/ldap/* > >>> + restorecon -v /var/lib/ldap/* > >>> + > >>> + service slapd start > >>> +or > >>> + systemctl start slapd.service > >> > >> Is all of this necessary to include here? I'm a detail-oriented > >> person, but I'm little concerned about how long these instructions are > >> (for DNS and LDAP set-up), since it gives the impression of a lot of > >> complexity (yes, I know, there are already a lot of moving parts > >> here). But one would hope that installing and configuring an LDAP > >> server would be simpler… > > > > That was the dilemma when I set out to do this. > > > > The aim of having a procedure to setup something completely standalone > > means that we need this stuff somewhere but it completely obscures the > > actual message of FedFS usage. > > > > The DNS configuration is fairly standard and much of the configuration > > would be present in any caching nameserver install so not a lot more is > > needed. > > > > But LDAP has proved to be rather painful over time and there are > > elements in the procedure that are useful for demonstrating those > > gotchas so it's worth keeping somewhere. > > This is the crux… LDAP is complicated, and the set up recipes depend > explicitly on which LDAP server is in use. That's exactly why I have > provided NSDB admin tools specifically for setting up the NCI, but I > agree that's not yet enough. Here's where we can use some thought > about how to make this a simpler process. > > For example, if we stick to keeping NSDB records under the LDAP > server's default domain controller naming context (dc=example,dc=net), > which should be created automatically by the LDAP server's installation > procedure, and not use a brand new naming context (o=fedfs) maybe there > is less LDAP specific set up required (i.e., we can manage more of the > set up with the server's default set up instructions, plus a generic > tool to turn it into an NSDB). OK, that sounds like a good idea, I'll give that a try. The other possibility, in order to retain the information, is to break it out into a separate howto for OpenLDAP "hints" perhaps. The same can be said of the DNS configuration steps, put that into a separate howto and then reference them in the preamble as assumptions about existing setup so that the actual FedFS procedure is clean and straight forward. > > Btw, your recipe may be missing a step: where is the FedFS schema > installed on the LDAP server? I'm trying to make that step unneeded by > having the LDAP vendors include the schema in their distribution > packaging, but that hasn't happened yet. Yeah, that's something that should have been left out of the slapd.conf above and added as a separate step. Ian
On Aug 29, 2012, at 10:50 PM, Ian Kent wrote: > On Wed, 2012-08-29 at 13:53 -0700, Chuck Lever wrote: >> On Aug 28, 2012, at 8:45 PM, Ian Kent wrote: >> >>> The thing that is still missing I think is usage of sub domains (either >>> delegated or included within an existing DNS configuration). But I don't >>> think we have a discovery protocol yet anyway. >> >> If I understand correctly, I don't think FedFS has "subdomains." >> Junctions can appear anywhere in the domain's namespace, in any >> export. They can even refer to other domains. But as far as I know >> there isn't the concept of a hierarchy of domains. The top level >> directory of /nfs4 is a flat namespace. > > I'm thinking of just the top level. > > There's nothing stopping someone setting up multiple name spaces. > > For example, assume you have a domain company.com and you've divided > your DNS up like: > > company.com > sales.company.com > engineering.company.com > > AFAICS each of these can have a name space accessed via: > > /nfs4/company.com > /nfs4/sales.company.com > /nfs4/engineering.company.com > > and these sub-domains could be actually delegated in the top level > domain or just prefixed names within the company.com DNS. Yes, you can do that. But perhaps a more natural way to do this with FedFS is to have the company.com FedFS domain root globally advertised (via SRV) and then put referrals in that domain's root to NFS servers that are administrated by the individual departments. So you have: /nfs4/company.com /nfs4/company.com/sales -> NFS servers in the sales department /nfs4/company.com/engineering -> NFS servers in the server department Each department can have its own NSDB, or they can share one NSDB that has multiple NCIs, or there can be a single FedFS administrator and one NSDB with one NCI for the whole domain. > I admit it isn't great to have the root domain and sub-domains at the > same level but I think this can be done and people with larger name > spaces will likely do it. > > Maybe there's an opportunity to think a bit about this before people > start asking for a hierarchic name spaces, although it is out of scope > for this task. Agree on both counts. This is definitely "user's guide" material, and the place we will have to draw our administrative use cases from is most likely going to be existing automounter deployments. -- Chuck Lever chuck[dot]lever[at]oracle[dot]com
Hi- On Aug 27, 2012, at 5:28 AM, Ian Kent wrote: > +Setup an NSDB (NameSpace DataBase) > +================================== > + > +1. Set parameters for NSDB connections: > + # nsdbparams(8) is used to set NSDB connection parameters > + nsdbparams update -e "dc=fedfs,dc=org" \ > + -D "cn=Manager,dc=fedfs,dc=org" \ > + zeus.fedfs.org > + > +2. Setup an OpenLDAP instance for fedfs.org: > + service slapd stop > +or > + systemctl stop slapd.service > + > +Create /etc/openldap/slapd.conf as: > + include /etc/openldap/schema/core.schema > + include /etc/openldap/schema/cosine.schema > + include /etc/openldap/schema/inetorgperson.schema > + include /etc/openldap/schema/nis.schema > + # Get this from the FedFS distribution > + include /etc/openldap/schema/fedfs.schema > + > + pidfile /var/run/openldap/slapd.pid > + argsfile /var/run/openldap/slapd.args > + > + database bdb > + suffix "dc=fedfs,dc=org" > + rootdn "cn=Manager,dc=fedfs,dc=org" > + rootpw secret > + directory /var/lib/ldap > + > + index objectClass eq,pres > + index ou,cn,mail,surname,givenname eq,pres,sub > + index uidNumber,gidNumber,loginShell eq,pres > + index uid,memberUid eq,pres,sub > + index nisMapName,nisMapEntry eq,pres,sub > + > +Create fedfs.org-naming-context.ldif as: > + dn: dc=fedfs,dc=org > + objectClass: domain > + dc: fedfs > + description: naming context > + > +Create an OpenLDAP instance for fedfs.org: > + # Ensure that /etc/openldap/ldap.conf is setup to use fedfs.org > + # by commenting out any other URI and BASE directives and then > + # add definitions to the end of the configuration. > + echo "URI ldap://zeus.fedfs.org/" >> /etc/openldap/ldap.conf > + echo "BASE dc=fedfs,dc=org" >> /etc/openldap/ldap.conf > + > + # Setup OpenLDAP for fedfs.org > + cd /etc/openldap > + restorecon -v slapd.conf > + > + rm -f /var/lib/ldap/* > + # supress warnings that this file does not exist > + touch /var/lib/ldap/DB_CONFIG > + > + rm -rf slapd.d > + slaptest -F slapd.d -f slapd.conf > + # I think this will restore the correct ownership > + # of the configuration directory tree but it may > + # be necessary to chmod -R ldap.ldap slapd.d also. > + restorecon -R -v slapd.d > + > + slapadd -l fedfs.org-naming-context.ldif > + chown ldap.ldap /var/lib/ldap/* > + restorecon -v /var/lib/ldap/* > + > + service slapd start > +or > + systemctl start slapd.service > + > +4. Add NCI (NSDB Container information) attributes to the > + naming context LDAP entry: > + nsdb-update-nci -l zeus.fedfs.org \ > + -D "cn=Manager,dc=fedfs,dc=org" \ > + -e "dc=fedfs,dc=org" I'm new to OpenLDAP, but you have inspired me to try it out as an NSDB. I used your instructions. It looks like OpenLDAP is in the middle of a major conversion from the old-school slapd.conf way of configuration to a new "OnLine Config" thingie. I think our instructions and tools should take advantage of the new method. We can build some simple tools that operate against a generic OpenLDAP install: 1. Add the FedFS schema via an ldapmodify command 2. Set up either an "o=fedfs" suffix and database, or add an "ou=fedfs" domain entry 3. We already have the nsdb-update-nci tool that can take it from there That might make all of this a lot easier. 1. and 2. would reside under the contrib/ directory in the fedfs-utils source tree. What do you think?
On Mon, 2012-09-24 at 13:40 -0400, Chuck Lever wrote: > Hi- > > On Aug 27, 2012, at 5:28 AM, Ian Kent wrote: > > > +Setup an NSDB (NameSpace DataBase) > > +================================== > > + > > +1. Set parameters for NSDB connections: > > + # nsdbparams(8) is used to set NSDB connection parameters > > + nsdbparams update -e "dc=fedfs,dc=org" \ > > + -D "cn=Manager,dc=fedfs,dc=org" \ > > + zeus.fedfs.org > > + > > +2. Setup an OpenLDAP instance for fedfs.org: > > + service slapd stop > > +or > > + systemctl stop slapd.service > > + > > +Create /etc/openldap/slapd.conf as: > > + include /etc/openldap/schema/core.schema > > + include /etc/openldap/schema/cosine.schema > > + include /etc/openldap/schema/inetorgperson.schema > > + include /etc/openldap/schema/nis.schema > > + # Get this from the FedFS distribution > > + include /etc/openldap/schema/fedfs.schema > > + > > + pidfile /var/run/openldap/slapd.pid > > + argsfile /var/run/openldap/slapd.args > > + > > + database bdb > > + suffix "dc=fedfs,dc=org" > > + rootdn "cn=Manager,dc=fedfs,dc=org" > > + rootpw secret > > + directory /var/lib/ldap > > + > > + index objectClass eq,pres > > + index ou,cn,mail,surname,givenname eq,pres,sub > > + index uidNumber,gidNumber,loginShell eq,pres > > + index uid,memberUid eq,pres,sub > > + index nisMapName,nisMapEntry eq,pres,sub > > + > > +Create fedfs.org-naming-context.ldif as: > > + dn: dc=fedfs,dc=org > > + objectClass: domain > > + dc: fedfs > > + description: naming context > > + > > +Create an OpenLDAP instance for fedfs.org: > > + # Ensure that /etc/openldap/ldap.conf is setup to use fedfs.org > > + # by commenting out any other URI and BASE directives and then > > + # add definitions to the end of the configuration. > > + echo "URI ldap://zeus.fedfs.org/" >> /etc/openldap/ldap.conf > > + echo "BASE dc=fedfs,dc=org" >> /etc/openldap/ldap.conf > > + > > + # Setup OpenLDAP for fedfs.org > > + cd /etc/openldap > > + restorecon -v slapd.conf > > + > > + rm -f /var/lib/ldap/* > > + # supress warnings that this file does not exist > > + touch /var/lib/ldap/DB_CONFIG > > + > > + rm -rf slapd.d > > + slaptest -F slapd.d -f slapd.conf > > + # I think this will restore the correct ownership > > + # of the configuration directory tree but it may > > + # be necessary to chmod -R ldap.ldap slapd.d also. > > + restorecon -R -v slapd.d > > + > > + slapadd -l fedfs.org-naming-context.ldif > > + chown ldap.ldap /var/lib/ldap/* > > + restorecon -v /var/lib/ldap/* > > + > > + service slapd start > > +or > > + systemctl start slapd.service > > + > > +4. Add NCI (NSDB Container information) attributes to the > > + naming context LDAP entry: > > + nsdb-update-nci -l zeus.fedfs.org \ > > + -D "cn=Manager,dc=fedfs,dc=org" \ > > + -e "dc=fedfs,dc=org" > > I'm new to OpenLDAP, but you have inspired me to try it out as an NSDB. > I used your instructions. > > It looks like OpenLDAP is in the middle of a major conversion from the > old-school slapd.conf way of configuration to a new "OnLine Config" > thingie. I think our instructions and tools should take advantage of > the new method. Yep, and has been for quite a while. > > We can build some simple tools that operate against a generic OpenLDAP > install: > > 1. Add the FedFS schema via an ldapmodify command I think I also had an example of that in one of the two things I wrote. > > 2. Set up either an "o=fedfs" suffix and database, or add an > "ou=fedfs" domain entry Yeah, if you want to define a new suffix and database for an example LDAP database I found it less problematic to blow away the existing database and recreate the directory based configuration from a slapd.conf. You can use ldapadd (offline add) or ldapmodify (online add) to make changes as you wish. Adding an additional domain to an existing tree with ldapmodify should work fine too. > > 3. We already have the nsdb-update-nci tool that can take it from > there Yep. > > That might make all of this a lot easier. 1. and 2. would reside under > the contrib/ directory in the fedfs-utils source tree. Yep, it would be good to capture this in the source tree. > > What do you think? It all sound good to me. I've got a bit of work on for a while now so the FedFS documentation task hasn't reached the top of the push down task stack quite yet. Ian
On Sep 24, 2012, at 7:05 PM, Ian Kent wrote: > On Mon, 2012-09-24 at 13:40 -0400, Chuck Lever wrote: >> Hi- >> >> On Aug 27, 2012, at 5:28 AM, Ian Kent wrote: >> >>> +Setup an NSDB (NameSpace DataBase) >>> +================================== >>> + >>> +1. Set parameters for NSDB connections: >>> + # nsdbparams(8) is used to set NSDB connection parameters >>> + nsdbparams update -e "dc=fedfs,dc=org" \ >>> + -D "cn=Manager,dc=fedfs,dc=org" \ >>> + zeus.fedfs.org >>> + >>> +2. Setup an OpenLDAP instance for fedfs.org: >>> + service slapd stop >>> +or >>> + systemctl stop slapd.service >>> + >>> +Create /etc/openldap/slapd.conf as: >>> + include /etc/openldap/schema/core.schema >>> + include /etc/openldap/schema/cosine.schema >>> + include /etc/openldap/schema/inetorgperson.schema >>> + include /etc/openldap/schema/nis.schema >>> + # Get this from the FedFS distribution >>> + include /etc/openldap/schema/fedfs.schema >>> + >>> + pidfile /var/run/openldap/slapd.pid >>> + argsfile /var/run/openldap/slapd.args >>> + >>> + database bdb >>> + suffix "dc=fedfs,dc=org" >>> + rootdn "cn=Manager,dc=fedfs,dc=org" >>> + rootpw secret >>> + directory /var/lib/ldap >>> + >>> + index objectClass eq,pres >>> + index ou,cn,mail,surname,givenname eq,pres,sub >>> + index uidNumber,gidNumber,loginShell eq,pres >>> + index uid,memberUid eq,pres,sub >>> + index nisMapName,nisMapEntry eq,pres,sub >>> + >>> +Create fedfs.org-naming-context.ldif as: >>> + dn: dc=fedfs,dc=org >>> + objectClass: domain >>> + dc: fedfs >>> + description: naming context >>> + >>> +Create an OpenLDAP instance for fedfs.org: >>> + # Ensure that /etc/openldap/ldap.conf is setup to use fedfs.org >>> + # by commenting out any other URI and BASE directives and then >>> + # add definitions to the end of the configuration. >>> + echo "URI ldap://zeus.fedfs.org/" >> /etc/openldap/ldap.conf >>> + echo "BASE dc=fedfs,dc=org" >> /etc/openldap/ldap.conf >>> + >>> + # Setup OpenLDAP for fedfs.org >>> + cd /etc/openldap >>> + restorecon -v slapd.conf >>> + >>> + rm -f /var/lib/ldap/* >>> + # supress warnings that this file does not exist >>> + touch /var/lib/ldap/DB_CONFIG >>> + >>> + rm -rf slapd.d >>> + slaptest -F slapd.d -f slapd.conf >>> + # I think this will restore the correct ownership >>> + # of the configuration directory tree but it may >>> + # be necessary to chmod -R ldap.ldap slapd.d also. >>> + restorecon -R -v slapd.d >>> + >>> + slapadd -l fedfs.org-naming-context.ldif >>> + chown ldap.ldap /var/lib/ldap/* >>> + restorecon -v /var/lib/ldap/* >>> + >>> + service slapd start >>> +or >>> + systemctl start slapd.service >>> + >>> +4. Add NCI (NSDB Container information) attributes to the >>> + naming context LDAP entry: >>> + nsdb-update-nci -l zeus.fedfs.org \ >>> + -D "cn=Manager,dc=fedfs,dc=org" \ >>> + -e "dc=fedfs,dc=org" >> >> I'm new to OpenLDAP, but you have inspired me to try it out as an NSDB. >> I used your instructions. >> >> It looks like OpenLDAP is in the middle of a major conversion from the >> old-school slapd.conf way of configuration to a new "OnLine Config" >> thingie. I think our instructions and tools should take advantage of >> the new method. > > Yep, and has been for quite a while. > >> >> We can build some simple tools that operate against a generic OpenLDAP >> install: >> >> 1. Add the FedFS schema via an ldapmodify command > > I think I also had an example of that in one of the two things I wrote. > >> >> 2. Set up either an "o=fedfs" suffix and database, or add an >> "ou=fedfs" domain entry > > Yeah, if you want to define a new suffix and database for an example > LDAP database I found it less problematic to blow away the existing > database and recreate the directory based configuration from a > slapd.conf. You can use ldapadd (offline add) or ldapmodify (online add) > to make changes as you wish. > > Adding an additional domain to an existing tree with ldapmodify should > work fine too. > >> >> 3. We already have the nsdb-update-nci tool that can take it from >> there > > Yep. > >> >> That might make all of this a lot easier. 1. and 2. would reside under >> the contrib/ directory in the fedfs-utils source tree. > > Yep, it would be good to capture this in the source tree. > >> >> What do you think? > > It all sound good to me. > I've got a bit of work on for a while now so the FedFS documentation > task hasn't reached the top of the push down task stack quite yet. There are benefits to putting this kind of documentation on a wiki, rather than including it in the fedfs-utils distribution itself. Inspired by your documentation post on this list, I've started playing with some stuff here: http://wiki.linux-nfs.org/wiki/index.php/FedFsInstallationGuides I've also worked out some scripts to go in contrib/ldap, which can be added to over time. I'm going to start posting patches for 0.9 very soon, hopefully next week. Again, some of this is going to boil down to picking one way out of the several ways of doing things. The "ou=fedfs,dc=example,dc=net" way has some interesting advantages over "o=fedfs": o Your FedFS domain name and the "dc=" suffix can match, making it easy for humans to find the NCE o You can have as many NCEs on an LDAP server as you have "dc=" suffixes o A "dc=" suffix is created by default by most LDAP server installation procedures, which makes NSDB set up a little simpler We're also going to have to start wrestling with ACIs and database index configuration on the NCE DIT. There was a Free IPA KDC/LDAP server at the NFS bake-a-thon this week. Soon we should start looking at similar scripts and documentation for setting up an NSDB on that server. -- Chuck Lever chuck[dot]lever[at]oracle[dot]com
diff --git a/doc/howto/fedfs-simple-setup-howto.txt b/doc/howto/fedfs-simple-setup-howto.txt new file mode 100644 index 0000000..6454f64 --- /dev/null +++ b/doc/howto/fedfs-simple-setup-howto.txt @@ -0,0 +1,465 @@ + +Simple example of basic FedFS setup and operation +================================================= + +The examples in this test procedure assume a Fedora 18 install. + +Assume that the package fedfs-utils-server and fedfs-utils-client +are installed on perseus.fedfs.org. And that fedfs-utils-server and +a nfs-utils built with fedfs-utils-devel installed on zeus.fedfs.org +and that fedfs-utils-lib is also installed on zeus.fedfs.org. + +We want to setup perseus.fedfs.org to provide the domain root, +be able to use FedFS to mount the domain root, and to setup and +use a FedFS juntion to access an export on zeus.fedfs.org upon +access to a directory within the domain root. And lastly, setup +autofs to use the FedFS program map to mount the root of the +domain. + +#Questions still to be answered: +#1. When and where (ie. on which server) is rpc.fedfsd used in the +# procedure below. +#2. What is the procedure for setting up a junction on a third +# unrelated machine. + +Setup a forwarding DNS server for fedfs.org +=========================================== + +Some of this setup may not be necessary but it is what I use to setup +DNS instances for testsing. Also many of the files created in step 1 +should be included in a caching nameserver package. Installing the +cacheing nameserver package and making appropriate modifications to +turn the cacheing nameserver into a forwarding nameserver iis really +all that needs to be done. Of course the fedfs zone files will always +be needed. + +The example assumes the private IP address range 192.168.1 is being +used and a nameserver that is able to satisfy name resolution for +external names is at 192.168.1.1 and the hosts perseus.fedfs.org and +zeus.fedfs.org are assigned addresses 192.168.1.32 and 192.168.1.31 +respectively. + +1. Setup zeus.fedfs.org as a forwarding DNS server: + + mkdir -p /var/named/master/fedfs.org + if [ ! -d /var/named/data ]; then + mkdir /var/named/data + fi + +Create /var/named/named.root using: + dig +norec NS > /var/named/named.root + +Create /etc/named.rfc1912.zones as: + // named.rfc1912.zones: + // + // ISC BIND named zone configuration for zones recommended by + // RFC 1912 section 4.1 : localhost TLDs and address zones + // + zone "localdomain" IN { + type master; + file "localdomain.zone"; + allow-update { none; }; + }; + + zone "localhost" IN { + type master; + file "localhost.zone"; + allow-update { none; }; + }; + + zone "0.0.127.in-addr.arpa" IN { + type master; + file "named.local"; + allow-update { none; }; + }; + + zone "1.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.0.ip6.arpa" IN { + type master; + file "named.ip6.local"; + allow-update { none; }; + }; + + zone "255.in-addr.arpa" IN { + type master; + file "named.broadcast"; + allow-update { none; }; + }; + + zone "0.in-addr.arpa" IN { + type master; + file "named.zero"; + allow-update { none; }; + }; + +Create /var/named/localdomain.zone as: + $TTL 86400 + @ IN SOA localhost root ( + 42 ; serial (d. adams) + 3H ; refresh + 15M ; retry + 1W ; expiry + 1D ) ; minimum + IN NS localhost + localhost IN A 127.0.0.1 + +Create /var/named/localhost.zone + $TTL 86400 + @ IN SOA @ root ( + 42 ; serial (d. adams) + 3H ; refresh + 15M ; retry + 1W ; expiry + 1D ) ; minimum + + IN NS @ + IN A 127.0.0.1 + IN AAAA ::1 + +Create /var/named/named.local as: + $TTL 86400 + @ IN SOA localhost. root.localhost. ( + 1997022700 ; Serial + 28800 ; Refresh + 14400 ; Retry + 3600000 ; Expire + 86400 ) ; Minimum + IN NS localhost. + 1 IN PTR localhost. + +Create /var/named/named.ip6.local as: + $TTL 86400 + @ IN SOA localhost. root.localhost. ( + 1997022700 ; Serial + 28800 ; Refresh + 14400 ; Retry + 3600000 ; Expire + 86400 ) ; Minimum + IN NS localhost. + 1 IN PTR localhost. + +Create /var/named/named.broadcast as: + $TTL 86400 + @ IN SOA localhost. root.localhost. ( + 42 ; serial (d. adams) + 3H ; refresh + 15M ; retry + 1W ; expiry + 1D ) ; minimum + IN NS localhost. + +Create /var/named/named.zero as: + $TTL 86400 + @ IN SOA localhost. root.localhost. ( + 42 ; serial (d. adams) + 3H ; refresh + 15M ; retry + 1W ; expiry + 1D ) ; minimum + IN NS localhost. + +Create /etc/named.conf as: + options { + listen-on port 53 { any; }; + listen-on-v6 port 53 { ::1; }; + directory "/var/named"; + allow-query { any; }; + forwarders { 192.168.1.1; }; + recursion yes; + }; + + logging { + channel default_debug { + file "data/named.run"; + severity dynamic; + }; + }; + + zone "." IN { + type hint; + file "named.root"; + }; + + include "/etc/named.rfc1912.zones"; + + include "/var/named/master/fedfs.org/zone.fedfs.org"; + +Create /var/named/master/fedfs.org/zone.fedfs.org as: + zone "fedfs.org" { + type master; + notify no; + file "master/fedfs.org/db.fedfs.org"; + }; + + zone "1.168.192.in-addr.arpa" { + type master; + notify no; + file "master/fedfs.org/db.192.168.1"; + }; + +Create /var/named/master/fedfs.org/db.fedfs.org as: + $TTL 3D + @ IN SOA ns.fedfs.org. hostmaster.fedfs.org. ( + 199802152 ; serial, todays date + todays serial # + 8H ; refresh, seconds + 2H ; retry, seconds + 4W ; expire, seconds + 1D ) ; minimum, seconds + ; + NS ns ; Inet Address of name server + ; + localhost A 127.0.0.1 + ns A 192.168.1.32 + zeus A 192.168.1.32 + perseus A 192.168.1.31 + + ; Assume these aren't present, they've been left to show an example + ; of the SRV record that would be used for a read-write FedFS domain + ; rather than the read-only setup we use here. + ;_nfs4._domainroot._tcp SRV 0 0 2049 perseus.fedfs.org. + ;_nfs4._write._domainroot._tcp SRV 0 0 2049 perseus.fedfs.org. + +Create /var/named/master/fedfs.org/db.192.168.1 as: + $TTL 3D + @ IN SOA ns.fedfs.org. hostmaster.fedfs.org. ( + 199802151 ; Serial, todays date + todays serial + 8H ; Refresh + 2H ; Retry + 4W ; Expire + 1D) ; Minimum TTL + NS ns.fedfs.org. + + 32 PTR ns.fedfs.org. + 32 PTR zeus.fedfs.org. + 31 PTR perseus.fedfs.org. + + service named start +or + systemctl start named.service + +2. On perseus.fedfs.org and zeus.fedfs.org configure /etc/resolve.conf: + cd /etc + cp resolve.conf resolv.conf.orig + echo "domain fedfs.org" > resolve.conf + echo "search fedfs.org" >> resolve.conf + echo "nameserver 192.168.1.32" >> resolve.conf + + +Setup an NSDB (NameSpace DataBase) +================================== + +1. Set parameters for NSDB connections: + # nsdbparams(8) is used to set NSDB connection parameters + nsdbparams update -e "dc=fedfs,dc=org" \ + -D "cn=Manager,dc=fedfs,dc=org" \ + zeus.fedfs.org + +2. Setup an OpenLDAP instance for fedfs.org: + service slapd stop +or + systemctl stop slapd.service + +Create /etc/openldap/slapd.conf as: + include /etc/openldap/schema/core.schema + include /etc/openldap/schema/cosine.schema + include /etc/openldap/schema/inetorgperson.schema + include /etc/openldap/schema/nis.schema + # Get this from the FedFS distribution + include /etc/openldap/schema/fedfs.schema + + pidfile /var/run/openldap/slapd.pid + argsfile /var/run/openldap/slapd.args + + database bdb + suffix "dc=fedfs,dc=org" + rootdn "cn=Manager,dc=fedfs,dc=org" + rootpw secret + directory /var/lib/ldap + + index objectClass eq,pres + index ou,cn,mail,surname,givenname eq,pres,sub + index uidNumber,gidNumber,loginShell eq,pres + index uid,memberUid eq,pres,sub + index nisMapName,nisMapEntry eq,pres,sub + +Create fedfs.org-naming-context.ldif as: + dn: dc=fedfs,dc=org + objectClass: domain + dc: fedfs + description: naming context + +Create an OpenLDAP instance for fedfs.org: + # Ensure that /etc/openldap/ldap.conf is setup to use fedfs.org + # by commenting out any other URI and BASE directives and then + # add definitions to the end of the configuration. + echo "URI ldap://zeus.fedfs.org/" >> /etc/openldap/ldap.conf + echo "BASE dc=fedfs,dc=org" >> /etc/openldap/ldap.conf + + # Setup OpenLDAP for fedfs.org + cd /etc/openldap + restorecon -v slapd.conf + + rm -f /var/lib/ldap/* + # supress warnings that this file does not exist + touch /var/lib/ldap/DB_CONFIG + + rm -rf slapd.d + slaptest -F slapd.d -f slapd.conf + # I think this will restore the correct ownership + # of the configuration directory tree but it may + # be necessary to chmod -R ldap.ldap slapd.d also. + restorecon -R -v slapd.d + + slapadd -l fedfs.org-naming-context.ldif + chown ldap.ldap /var/lib/ldap/* + restorecon -v /var/lib/ldap/* + + service slapd start +or + systemctl start slapd.service + +4. Add NCI (NSDB Container information) attributes to the + naming context LDAP entry: + nsdb-update-nci -l zeus.fedfs.org \ + -D "cn=Manager,dc=fedfs,dc=org" \ + -e "dc=fedfs,dc=org" + + +Add a FedFS junction within a domain root directory +=================================================== + +Assuming there is a file system mounted on /vm (or just a directory +we can export) on server zues.fedfs.org which we want to access +under the domain root as <domain root mount point>/vm, we will be +exporting /.domainroot-fedfs.org as the domain root. + +1. Add an entry to /etc/exports on zeus.fedfs.org: + + # Add to /etc/exports + /vm *(ro) + + # Restart the nfs service or just re-export the table + exportfs -r + +2. Add a junction to the domain root on persues.fedfs.org: + + # + # Tell nfsref the LDAP server (the NSDB) we are using to + # record file system name (FSN) and file system location + # (FSL) uuids. This assumes the LDAP connection parameters + # have been setup as in step 1 of "Setup an NSDB". + # + export FEDFS_NSDB_HOST=zeus.fedfs.org + + # + # Add the junction metadata to the directory and update + # the NSDB with uuid info of the junction. + # + mkdir -p /.domainroot-fedfs.org/vm + nfsref --type=nfs-fedfs \ + add /.domainroot-fedfs.org/vm \ + zeus.fedfs.org /vm + + +Setup fedfs domain root export (read-only case) +=============================================== + +For this we are seeking to mount the domain root exported from host +perseus.fedfs.org. + +1. Add an SRV record for the FedFS file server to DNS: + + _nfs4._domainroot._tcp SRV 0 0 2049 perseus.fedfs.org. + +2. Restart named to make in available. + + service named restart +or + systemctl restart named.service + +3. Add an entry to /etc/exports on perseus.autofs.test: + + # + # Created when we added the junction above. + # mkdir /.domainroot-fedfs.org + # + /.domainroot-fedfs.org *(ro) + +4. Restart NFS: + + service nfs restart +or + systemctl restart nfs.service + +5. Mount using the FedFS mount utility on a local directory: + + mount -v -t fedfs /nfs4/fedfs.org /mnt + mount | grep domainroot + perseus.fedfs.org:/.domainroot-fedfs.org/ on /mnt type nfs4 ... + + cd /mnt/vm + + # + # This check assumes /etc/mtab is symlinked to /proc/mounts + # as it is in Fedora. Kernel automounted file systems will + # not be present in the text based /etc/mtab and so will not + # be seen in it. Look to /proc/mounts instead in this case. + # + mount | grep ^zeus.fedfs.org.*vm + zeus.fedfs.org:/vm/ on /mnt/vm type nfs4 ... + + # Ha, move out of the directory so it can be umounted + cd + +5. Lastly cleanup: + + # + # This example includes a specific umount of the junction + # (/mnt/vm) but such kernel automounted file systems are + # umounted automatically (when they are not in use) so it + # may not be present when this step is done. + # + umount /mnt/vm + umount /mnt + +Setup autofs to automount the domain root +========================================= + +1. Add a line to /etc/auto.master to automount FedFS root domains: + + # + # Note that the autofs pseudo option "nobind" probably + # should be used. In the case here it is required because + # the FedFS client also hosts the root of the domain and + # autofs will see the mount is local and perform a bind + # mount instead of an NFS mount. That, of course, means + # file system lookups won't be with an NFS file system + # so NFS referals can't be followed. + # + # Also note that the autofs mount point name must be + # /nfs4 to be able to mount nfs4 root domains. + # + echo "/nfs4 /usr/sbin/fedfs-map-nfs4 nobind" >> /etc/auto.master + +2. Restart or reload the autofs service: + + service autofs restart +or + systemctl restart autofs.service + +3. Check that we can mount the domain root and the referal: + + # automount the root domain. + [raven@perseus ~]$ ls /nfs4/fedfs.org + top.txt vm + + # automount the referal (from a different machine). + [raven@perseus ~]$ ls /nfs4/fedfs.org/vm + lost+found test.txt + + # Check they were mounted. + [raven@perseus ~]$ mount |grep perseus|grep nfs4 + perseus.fedfs.org:/.domainroot-fedfs.org/ on /nfs4/fedfs.org type nfs4 ... + [raven@perseus ~]$ mount |grep zeus|grep nfs4 + zeus.fedfs.org:/vm/ on /nfs4/fedfs.org/vm type nfs4 ... + +
Add a howto to setup a DNS domain fedfs.org, an (OpenLDAP) LDAP instance with context dc=fedfs,dc=org and work through the steps to setup a read-only FedFS root domain with a single referral. Signed-off-by: Ian Kent <raven@themaw.net> --- doc/howto/fedfs-simple-setup-howto.txt | 465 ++++++++++++++++++++++++++++++++ 1 files changed, 465 insertions(+), 0 deletions(-) create mode 100644 doc/howto/fedfs-simple-setup-howto.txt