Bull GNU/Linux NFSv4 project

2.6.17-rc2-CITI_NFS4_ALL-1 performances and regression tests

Tests performed

Here are the last results of testing  the kernel and CITI patch: linux-2.6.17-rc2-CITI_NFS4_ALL-1 with the following tools:

Test #1 : connectathon 04 (update 2003/12) (ltp-full-20060615)  LTP

• mode extended (-t) is used

Test #2 : nfs_fsstress (ltp-full-20060615)  LTP

• Tests are done with various number of subprocessess : 2, 4, 8, 16, 32

• Tests are done with various length of instructions lists: 1, 2, 4, 8, 16, 32, 64, 128

Test #3 : fsx (ltp-full-20060615) LTP

• fsx is run with 50000 iterations
  

Test #4 : Iozone (version 3.217) HERE


• Run iozone -ace -r 32 -i 0 -i 1 -U mountpoint

Test #5 : ffsb (version 5.2.1) FFSB

• Run with default profile_stress_test file as profile

Test #6 : locks testing   HERE


• Run mode local done with 500 processes

• Run  mode network done with two clients , 500 processes each

Test #7 : dbench (3.04)  HERE


• Run with 10, 100, 1000 clients

Test #8 : ACL testing (ltp-full-20060615) LTP

Performances Results

• We run iozone (Test#4) on kernel 2.6.17-rc2 keeping  the same kernel  and hardware configuration in order to compare:
  • read and write performance throughputs
    
  • asynchronous and synchronous modes of the command mount ( async exports used in both cases)
    
  • nfsv4 and nfsv3 (tcp and udp) and samba3.0.3
  
  

  The following parameters were used:  iozone -+q 5  -ace  -r 32  -i 0  -i 1  -f /mnt/nosec/file  -U /mnt/nosec

  mount options rsize,wsize were set to 32768

  Kernel .config file impact is very important on performances. Check it when bad performances.
  For example, CONFIG_DEBUG_SLAB set will drastically decrease the performances.

• The  results on reads are HERE
Remarks:
• on nfs:
nfsv3 udp is lightly better and almost reaches the maximum bandwidth of the link (1Gbits) except for 64KBytes files for which tcp nfsv4 is better.
nfsv4 tcp and nfsv3 tcp are similar


• on samba:
The configuration used is the default one like for nfsv4. Transfer buffer size and reduced asynchronous capabilities of samba3 are among the reasons  of its poor performance. Other parameters than the default ones have to be set and tuned to evaluate more accurately the difference of the performance between NFSV4 and Samba


• The results on writes are  HERE
Remarks:
• on async mode (command mount)
if nfsv3 udp is lightly better than nfsv4 tcp this one is better than nfsv3 tcp which has an issue to study around 128KBytes-2048KBytes files.

Robustness Results

• The objective was to run:
• Tests #1 #2 #3 #4 #5 #6 #7 alone for: 2 hours, 15 hours, 60 hours.
• Tests #1 #2 #3 #4 #5 were also run all together simultaneously for: 2 hours , 15 hours , 60 hours

• Asynchronous mode is used for both: mount command and export option
  

• The results are HERE
Remarks:
• Tests #4 alone (option -U used)
  

• Tests #6

• Tests #1 #2 #3 #4 #5 together 60 hours

With Kerberos Performances Results

• iozone (Test#4) was also used to compare performance on reads and writes using :
• the different flavors krb5, krb5i, krb5p
  
• asynchronous and synchronous modes of the command mount (async exports used in both cases)

• The following parameters were used:
  

mount options rsize,wsize were set to 32768

• The results on async reads are : HERE
Remarks:

We retrieve logically a degradation following the flavor used. Nevertheless if for krb5i there is space to try to improve to be closer to the krb5 performance we have clearly a performance bug for krb5p (BUGZILLA#117).
Some patchs are currently in study for fixing that.

• The results on sync reads are: HERE

• The results on async writes are: HERE
  

• The results on sync writes are: HERE

With  Kerberos Robustness Results

• The objective was  to run :
• Tests #1 #2 #3 #4 #5  alone for: 2 hours, 15 hours, 60 hours using the different flavors krb5, krb5i , krb5p   for 2 hours, 15 hours, 60 hours
• Tests #1 #2 #3 #4 #5 were also run all together simultaneously for: 2 hours , 15 hours , 60 hours
  

Asynchronous mode is used for both: mount command and export option


• The results are HERE
Remarks:
With the flavor krb5i we got an issue (BUGZILLA #109) dealing with memory consumption. It happens when running test #1 #2 #3 #4 #5 all together but I reproduce it when only using connectathon in a while loop.

WAN Results

Tests have been started between a machine at Bull and a machine at OSDL.
The characteristics of this link has been also reproduced using NETEM between two machines at Bull.
Here are the RESULTS.

Interoperability summary

Kerberos Summary

			Servers
		ia32	powerpc64	x86_64
	ia32	krb5,krb5i,krb5p done	TBD	TBD
Clients	powerpc64	TBD	TBD	TBD
	x86_64	TBD	TBD	TBD

Software versions

Hardware configuration

Client and Server

• 2 processors : Intel(R)
• Xeon(TM) CPU 2.80GHz, cache 512 KB
• Total memory: 2Gb
• Ethernet: 1Gb/s link
• Distribution : Fedora Core
  
• Kerberos V5 MIT1.3.1-6

Conclusion

Core linux-2.6.17-rc2-CITI_NFS4_ALL-1 functions should be considered as stable without kerberos. About robustness and performances, it is at least at the same level than nfsv3 and sometimes better than nfsv3 tcp. Nevertheless some bugs registered in BUGZILLA are still under investigation (bugs #114, #110, #113). It integrates now the security flavors krb5, krb5i, krb5p (krb5p was missing in 2.6.16-CITI_NFS4_ALL-2) but has two important issues to resolve # 117, #109.