Oracle DNFS

實驗說明：
服務器一：TEST1( NFS服務端)
服務器二：TEST2( NFS客戶端)

其中：
NFS服務端將 /vol/oradata目錄設置為共享目錄；
NFS客戶端上的數據庫orcl使用共享目錄/vol/oradata，并在共享目錄里創建表空間，生成的數據文件自動保存在TES1(NFS服務端)機器上；

(1)檢查NFS服務端nfs和portmap服務正常啟動
TEST1(NFS服務端)：
[root@TEST1 ~]# /etc/init.d/nfs status
rpc.mountd (pid 4123) is running...
nfsd (pid 4120 4119 4118 4117 4116 4115 4114 4113) is running...
rpc.rquotad (pid 4091) is running...
[root@TEST1 ~]# /etc/init.d/portmap status
portmap (pid 3585) is running...

(2)配置exports文件，共享/vol/oradata目錄
[root@TEST1 ~]# cat /etc/exports 
/vol/oradata *(rw,sync,no_wdelay,insecure,insecure_locks,no_root_squash)

(3)NFS客戶端通過mount命令將共享目錄/vol/oradata掛載到/u02/oradata/prod1目錄
TEST2（NFS客戶端）：
[root@TEST2 ~]# mount -o remount /vol/oradata /u02/oradata/prod1
[root@TEST2 ~]# df -h
Filesystem            Size  Used Avail Use% Mounted on
/dev/sda3              15G  6.0G  7.9G  44% /
/dev/sda2              30G  8.6G   20G  31% /u01
/dev/sda1              99M   29M   65M  31% /boot
tmpfs                 1.5G  540M  982M  36% /dev/shm
TEST1.example.com:/vol/oradata
                      433G   30G  381G   8% /u02/oradata/prod1

(4)TEST2(NFS客戶端)配置DNFS
[root@TEST2 ~]# vi /etc/oranfstab
server:server
path:TEST1.example.com
local:TEST2.example.com
export:/vol/oradata mount:/u02/oradata/prod1

(5)TEST2(NFS客戶端)啟用DNFS客戶端
[oracle@TEST2 ~]$ cd /u01/app/oracle/product/11.2.0/dbhome_1/rdbms/lib/
[oracle@TEST2 lib]$ make -f ins_rdbms.mk dnfs_on
rm -f /u01/app/oracle/product/11.2.0/dbhome_1/lib/libodm11.so; cp /u01/app/oracle/product/11.2.0/dbhome_1/lib/libnfsodm11.so /u01/app/oracle/product/11.2.0/dbhome_1/lib/libodm11.so

(6)TEST2(NFS客戶端)在共享目錄下創建表空間test_tbs;
SQL> create tablespace test_tbs datafile '/u02/oradata/prod1/test_tbs01.dbf' size 10M;
Tablespace created.

(7) 驗證
如果在TEST2客戶端/u02/oradata/prod1目錄創建的數據文件自動生成在TEST1服務端/vol/oradata目錄下，說明DNFS成功；
TES1服務端查看文件
[root@TEST1 ~]# ll -rth /vol/oradata/test_tbs01.dbf 
-rw-r----- 1 oracle oinstall 11M Jul 19 12:40 /vol/oradata/test_tbs01.dbf
TEST2客戶端查看文件
[oracle@TEST2 lib]$ ll -rth /u02/oradata/prod1/test_tbs01.dbf 
-rw-r----- 1 oracle oinstall 11M Jul 19 12:40 /u02/oradata/prod1/test_tbs01.dbf

查看以下動態性能視圖
如果結果為空，需要重啟數據庫實例；
SQL> set linesize 200
SQL> col filename for a50
SQL> select * from v$dnfs_files;
FILENAME                                             FILESIZE       PNUM     SVR_ID
-------------------------------------------------- ---------- ---------- ----------
/u02/oradata/prod1/test_tbs01.dbf                    10493952         10          1

SQL> col svrname for a20
SQL> col dirname for a20
SQL> select * from v$dnfs_servers;
        ID SVRNAME              DIRNAME                 MNTPORT    NFSPORT      WTMAX      RTMAX
---------- -------------------- -------------------- ---------- ---------- ---------- ----------
         1 server               /vol/oradata                963       2049      32768      32768
 

關于DNFS更詳細的內容，參見官方文檔，鏈接如下：
http://docs.oracle.com/cd/E11882_01/install.112/e47689/post_inst_task.htm#LADBI1297
http://docs.oracle.com/cd/E11882_01/install.112/e41961/storage.htm#CWLIN279

一：Enabling a Direct NFS Client
By default Direct NFS Client serves mount entries found in /etc/mtab. 
No other configuration is required. 
You can use oranfstab to specify additional Oracle Database specific options to Direct NFS Client. 
For example, you can use oranfstab to specify additional paths for a mount point.

A new Oracle Database specific file oranfstab can be added to either /etc or to $ORACLE_HOME/dbs. 
When oranfstab is placed in $ORACLE_HOME/dbs, its entries are specific to a single database. 
However, when oranfstab is placed in /etc, then it is global to all Oracle databases, and hence can contain mount points for all Oracle databases.

Note:
Direct NFS Client does not work and falls back to the traditional kernel NFS path if the back-end NFS server does not support a write size (wtmax) of 32768 or larger.

Direct NFS Client determines mount point settings to NFS storage devices based on the configurations in /etc/mtab. Direct NFS Client looks for the mount point entries in the following order:
(1) $ORACLE_HOME/dbs/oranfstab
(2) /etc/oranfstab
(3) /etc/mtab
It uses the first matched entry as the mount point.

Oracle Database requires that mount points be mounted by the kernel NFS system even when served through Direct NFS Client.

Complete the following procedure to enable Direct NFS Client:
1.You can optionally create an oranfstab file with the following attributes for each NFS server to be accessed using Direct NFS Client:

(1)Server: 
The NFS server name.
(2)Path: 
Up to four network paths to the NFS server, specified either by IP address, or by name, as displayed using the ifconfig command on the filer.
(3)Local: 
Up to four local paths on the database host, specified by IP address or by name, as displayed using the ifconfig command run on the database host.
(4)Export: 
The exported path from the NFS server.
(5)Mount: 
The corresponding local mount point for the exported volume.
(6)Dontroute: 
Specifies that outgoing messages should not be routed by the operating system, but sent using the IP address they are bound to. 
Please note that this attribute does not work on Linux with multiple paths in the same subnet.
(7)mnt_timeout: 
Specifies (in seconds) the time for which Direct NFS Client should wait for a successful mount before timing out. 
This parameter is optional and the default timeout is 10 minutes.
(8)management: 
Enables Direct NFS Client to use the management interface for SNMP queries. 
You can use this parameter if SNMP is running on separate management interfaces on the NFS server. 
The default value is the server parameter value.
(9)community: 
Specifies the community string for use in SNMP queries. The default value is public.

Example 3-1 Using Local and Path NFS Server Entries

The following example uses both local and path. Since they are in different subnets, we do not have to specify dontroute.

server: MyDataServer1
local: 192.0.2.0
path: 192.0.2.1
local: 192.0.100.0
path: 192.0.100.1
export: /vol/oradata1 mount: /mnt/oradata1
community: private

Example 3-2 Using Local and Path in the Same Subnet, with dontroute

The following example shows local and path in the same subnet. dontroute is specified in this case:

server: MyDataServer2
local: 192.0.2.0
path: 192.0.2.128
local: 192.0.2.1
path: 192.0.2.129
dontroute
export: /vol/oradata2 mount: /mnt/oradata2
management: 192.0.10.128

Example 3-3 Using Names in Place of IP Addresses, with Multiple Exports

server: MyDataServer3
local: LocalPath2
path: NfsPath2
local: LocalPath3
path: NfsPath3
local: LocalPath4
path: NfsPath4
local: LocalPath5
path: NfsPath5
dontroute
export: /vol/oradata3 mount: /mnt/oradata3
export: /vol/oradata4 mount: /mnt/oradata4
export: /vol/oradata5 mount: /mnt/oradata5
export: /vol/oradata6 mount: /mnt/oradata6

By default, Direct NFS Client is installed in a disabled state. To enable Direct NFS Client, complete the following steps on each node. If you use a shared Grid home for the cluster, then complete the following steps in the shared Grid home:

Log in as the Oracle Grid Infrastructure installation owner.
Change directory to Grid_home/rdbms/lib.
Enter the following commands:
$ make -f ins_rdbms.mk dnfs_on

二： Disabling Direct NFS Client
Complete the following steps to disable the Direct NFS Client:

Log in as the Oracle software installation owner, and disable Direct NFS Client using the following commands:

cd $ORACLE_HOME/rdbms/lib
make -f ins_rdbms.mk dnfs_off
Remove the oranfstab file.

Note:
If you remove an NFS path that Oracle Database is using, then you must restart the database for the change to take effect.

歡迎關注我的微信公眾號"IT小Chen"，共同學習，共同成長！！！