Tell me more ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.
up vote 2 down vote favorite
1

I interconnect several of my Linux servers with NFS. For a while its been going fine but for some reason keeps getting more and more unstable

Example: Copy a 4 GB file from client to NFS server, fails before finishing with an IO error. dmesg is filled with 

[1525450.884071] nfs: server quackron not responding, timed out
[1525451.384072] nfs: server quackron not responding, timed out
[1525451.884072] nfs: server quackron not responding, timed out
[1525452.384086] nfs: server quackron not responding, timed out
[1525452.884065] nfs: server quackron not responding, timed out
[1525584.112127] nfs: server quackron not responding, timed out
[1525669.492066] nfs: server quackron not responding, timed out
[1525930.544062] nfs: server quackron not responding, timed out

Mount options (the many extra options are to prevent a disappearing NFS server from locking the whole directory until a restart or umount trickery)

rw,intr,soft,timeo=5,retrans=5,actimeo=10,retry=5,vers=4,addr=10.0.0.20,clientaddr=10.0.0.1

Going to the server, I get this in dmesg

[1384141.237197] NFSD: Using /var/lib/nfs/v4recovery as the NFSv4 state recovery directory
[1384141.237237] NFSD: starting 90-second grace period
[1439491.137710] nfsd: last server has exited, flushing export cache
[1439492.461197] NFSD: Using /var/lib/nfs/v4recovery as the NFSv4 state recovery directory
[1439492.461236] NFSD: starting 90-second grace period

I can't find it in this dmesg or the log, but I do know I've seen "nfsd last server has exited signal 15" errors. Googling can't seem to find any useful information

Any idea's what would cause nfs to constantly crash?

Note: This server is running Ubuntu Server 11.10 and is fully up to date

share|improve this question
Have you tried scp to copy the same file? – Karlson Feb 23 '12 at 21:29
@Karlson Just tried it. Works perfectly fine. Right afterwards tried NFS just crashed again – TheLQ Feb 24 '12 at 21:16
@TheLQ - Please input the defined NFS export from /etc/exports. – 2bc Mar 23 '12 at 0:28
Its not in /etc/exports, its mounted in a script for various reasons. Script command: sudo mount -t nfs -o intr,soft,timeo=5,retrans=5,actimeo=10,retry=5 -v -v -v $1:$2 $3 with the variables at the end corresponding to server, remote folder, and local folder – TheLQ Mar 23 '12 at 1:33
is this on 32 or on 64 bits ? – Coren Mar 23 '12 at 7:56
show 6 more comments

3 Answers

up vote 1 down vote accepted
+50

..adding to an answer instead of just comment

modify your mount on the client side to: 

sudo mount -t nfs -o intr,hard,timeo=14,rsize=8192,wsize=8192

or if you put it in fstab on the client side 

remoteserver:/remote/path /local/path nfs rsize=8192,wsize=8192,hard,time0=14,intr 0 0

as a test.

I use these settings at work and we routinely transfer several TB a week.

Use these settings adding none of the others except for your specific pathing variables.

?any luck?

share|improve this answer
Just did some testing, seems to of fixed the problem! Connection is stable enough to copy a 3.6 GB ISO without disconnecting. Seems to even be a bit faster, although I'm not for sure. Thanks! – TheLQ Mar 26 '12 at 21:13
Awesome!! You can play with the rsize and wsize this modifies the data transfer buffer sizes. The only thing you need to worry about is your network setup as it relates to MTU I have never had an issue but, others have. Without going and looking I think you can go up to 32,xxx for them. – 2bc Mar 26 '12 at 21:22
up vote 0 down vote

If NFS is that troublesome and scp works without problem - use sshfs.

share|improve this answer
Except the reason I chose NFS was because these are old desktop computers, and sshfs with large file transfers take up too many resources. And yes, I've benchmarked this and had the system setup for 2 months with sshfs, and it was very slow – TheLQ Mar 23 '12 at 23:41
Did you enable/disable compression on sshfs? Or are we talking about "very old" - less than 500 MHz computers? – Nils Mar 24 '12 at 20:53
The server that mounts all the shares has a 1 GHz Pentium 3 processor (yea, I know). Even with no compression and arcfour encryption it was still noticeably slow. Besides, this isn't a fix, this is a workaround. I'm really trying to figure out why NFS won't work, not how to work around it – TheLQ Mar 24 '12 at 21:49
up vote 0 down vote

If it's a dedicated NFS VLAN, double check you don't have any duplicate IPs on your network. I just debugged an issue very much like this, and it turns out there were some typos in kickstart files and out of 40 hosts, we had 5 that were using duplicate IP addresses. NFS would work great and copies would be fast for a while, but performance would gradually taper off, then hang altogether.

share|improve this answer
Went ahead and verified that there was no duplicate IP's, couldn't find any – TheLQ Mar 26 '12 at 21:12

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.