Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

I interconnect several of my Linux servers with NFS. For a while its been going fine but for some reason keeps getting more and more unstable

Example: Copy a 4 GB file from client to NFS server, fails before finishing with an IO error. dmesg is filled with

[1525450.884071] nfs: server quackron not responding, timed out
[1525451.384072] nfs: server quackron not responding, timed out
[1525451.884072] nfs: server quackron not responding, timed out
[1525452.384086] nfs: server quackron not responding, timed out
[1525452.884065] nfs: server quackron not responding, timed out
[1525584.112127] nfs: server quackron not responding, timed out
[1525669.492066] nfs: server quackron not responding, timed out
[1525930.544062] nfs: server quackron not responding, timed out

Mount options (the many extra options are to prevent a disappearing NFS server from locking the whole directory until a restart or umount trickery)

rw,intr,soft,timeo=5,retrans=5,actimeo=10,retry=5,vers=4,addr=10.0.0.20,clientaddr=10.0.0.1

Going to the server, I get this in dmesg

[1384141.237197] NFSD: Using /var/lib/nfs/v4recovery as the NFSv4 state recovery directory
[1384141.237237] NFSD: starting 90-second grace period
[1439491.137710] nfsd: last server has exited, flushing export cache
[1439492.461197] NFSD: Using /var/lib/nfs/v4recovery as the NFSv4 state recovery directory
[1439492.461236] NFSD: starting 90-second grace period

I can't find it in this dmesg or the log, but I do know I've seen "nfsd last server has exited signal 15" errors. Googling can't seem to find any useful information

Any idea's what would cause nfs to constantly crash?

Note: This server is running Ubuntu Server 11.10 and is fully up to date

share|improve this question
    
Have you tried scp to copy the same file? –  Karlson Feb 23 '12 at 21:29
    
@Karlson Just tried it. Works perfectly fine. Right afterwards tried NFS just crashed again –  TheLQ Feb 24 '12 at 21:16
    
@TheLQ - Please input the defined NFS export from /etc/exports. –  2bc Mar 23 '12 at 0:28
    
Its not in /etc/exports, its mounted in a script for various reasons. Script command: sudo mount -t nfs -o intr,soft,timeo=5,retrans=5,actimeo=10,retry=5 -v -v -v $1:$2 $3 with the variables at the end corresponding to server, remote folder, and local folder –  TheLQ Mar 23 '12 at 1:33
    
is this on 32 or on 64 bits ? –  Coren Mar 23 '12 at 7:56
show 6 more comments

5 Answers

up vote 1 down vote accepted
+50

..adding to an answer instead of just comment

modify your mount on the client side to:

sudo mount -t nfs -o intr,hard,timeo=14,rsize=8192,wsize=8192 

or if you put it in fstab on the client side

remoteserver:/remote/path /local/path nfs rsize=8192,wsize=8192,hard,time0=14,intr 0 0 

as a test.

I use these settings at work and we routinely transfer several TB a week.

Use these settings adding none of the others except for your specific pathing variables.

?any luck?

share|improve this answer
    
Just did some testing, seems to of fixed the problem! Connection is stable enough to copy a 3.6 GB ISO without disconnecting. Seems to even be a bit faster, although I'm not for sure. Thanks! –  TheLQ Mar 26 '12 at 21:13
    
Awesome!! You can play with the rsize and wsize this modifies the data transfer buffer sizes. The only thing you need to worry about is your network setup as it relates to MTU I have never had an issue but, others have. Without going and looking I think you can go up to 32,xxx for them. –  2bc Mar 26 '12 at 21:22
add comment

If NFS is that troublesome and scp works without problem - use sshfs.

share|improve this answer
    
Except the reason I chose NFS was because these are old desktop computers, and sshfs with large file transfers take up too many resources. And yes, I've benchmarked this and had the system setup for 2 months with sshfs, and it was very slow –  TheLQ Mar 23 '12 at 23:41
    
Did you enable/disable compression on sshfs? Or are we talking about "very old" - less than 500 MHz computers? –  Nils Mar 24 '12 at 20:53
    
The server that mounts all the shares has a 1 GHz Pentium 3 processor (yea, I know). Even with no compression and arcfour encryption it was still noticeably slow. Besides, this isn't a fix, this is a workaround. I'm really trying to figure out why NFS won't work, not how to work around it –  TheLQ Mar 24 '12 at 21:49
add comment

If it's a dedicated NFS VLAN, double check you don't have any duplicate IPs on your network. I just debugged an issue very much like this, and it turns out there were some typos in kickstart files and out of 40 hosts, we had 5 that were using duplicate IP addresses. NFS would work great and copies would be fast for a while, but performance would gradually taper off, then hang altogether.

share|improve this answer
    
Went ahead and verified that there was no duplicate IP's, couldn't find any –  TheLQ Mar 26 '12 at 21:12
add comment

I'm having the same issue. This looks like a bug in the way it gets mounted at boot time. Put noauto in /etc/fstab so the NFS volume does not get mounted at boot time. Once the machine has been up, manually mount it. OR just put mount -a in /etc/rc.local. Might be some bad bugs in upstart and udev... makes no sense!

share|improve this answer
    
This was mounted inside of a script for various reasons –  TheLQ Jul 6 '13 at 12:58
add comment

I got a series of error:

[26017.155508] svc: failed to register nfsdv2 RPC service (errno 111).
[26017.155555] svc: failed to register nfsaclv2 RPC service (errno 111).
[26017.155786] nfsd: last server has exited, flushing export cache
[26017.156528] nfsd: last server has exited, flushing export cache
[26026.042279] svc: failed to register nfsdv2 RPC service (errno 111).
[26026.042332] svc: failed to register nfsaclv2 RPC service (errno 111).
[26026.042573] nfsd: last server has exited, flushing export cache

and it was resolved through "/etc/init.d/portmap restart", not sure if it help?

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.