Tag Info

Hot answers tagged

18

Here's another way to do locking in shell script that can prevent the race condition you describe above, where two jobs may both pass line 3. The noclobber option will work in ksh and bash. Don't use set noclobber because you shouldn't be scripting in csh/tcsh. ;) lockfile=/var/tmp/mylock if ( set -o noclobber; echo "$$" > "$lockfile") 2> ...


8

I'm pretty sure that you are looking at legacy concerns. Recall that the Perl5 manual was released in 1994 and that it was just an edit of Perl4's manual from 1991. In those days it could probably be said about the oft-named Nightmare File System that "it isn't how well the bear dances that amazes, but that it dances at all". NFS2 in the 1991 epoch was ...


8

NFS was designed with the idea that user and group ids would be the same on all machines across the network. For ordinary users, that works ok. But root's UID is always 0, and just because you have root on one box, it doesn't mean that you should have root access to every machine on the network. Therefore, NFS treats root specially. By default, root is ...


8

Mount the NFS-share on the clients using the mount-options "bg,intr,hard". Most important in your case is "bg" for background - which tells the system not to block when the server is not available. "intr" for interrruptable - so you can kill hanging mounts on the client with the kill command. "hard" is the opposite of "soft". The difference is that "hard" ...


7

Remember that each of the NFS client systems will determine the username by looking up the numerical UID in /etc/passwd, or your centralized user database. The NFS server only stores the UID in numerical format, and does not know about usernames. This is also true for group names vs. GIDs. In your case, serverA and serverB must have different usernames ...


7

Olaf Kirch originally developed both the user space and kernel based version of the NFS server. In his year 2000 book, "Linux Network Administration" he says: The 2.2.0 kernel supports an experimental kernel-based NFS server developed by Olaf Kirch and further developed by H.J. Lu, G. Allan Morris, and Trond Myklebust. The kernel-based NFS support provides ...


6

Bad form, I know, to answer my own question, but.... I needed a couple more steps, outlined here. In short, I needed to execute: sudo nfsd update As another detail, I added the client name to the export and removed the "-rw" flag.


6

If you use NFSv4 with sec=krb5p, then yes, it is secure. (That means use Kerberos 5 for authentication, and encrypt the connection for privacy.) But if you use NFS v3 or NFS v4 with sys=system, then no, it's not secure at all. There might also be some concern with exposing the kerberos and rpc ports to the internet at large, just in case of unknown ...


6

No. You can export a device file through NFS or some other network filesystems. But the meaning of the device file is dependent on the machine where you open it. If you export /dev/video0 over NFS from a server machine to a client machine, the client machine just sees “character device 81:0”, and interprets it as its own video capture device. The client ...


6

Check these items, and see if any work for you: On the client, if you're not already using the cto option in the options column of the /etc/fstab line for your NFS filesystem, add it. cto tells the nfs client to open files via close-to-open, which makes them refresh the file whenever they open it. On the server, make sure your filesystem is exported with ...


6

The quick way The quickest way to transfer files over a LAN is likely not rsync, unless there are few changes. rsync spends a fair bit of time doing checksums, calculating differences, etc. If you know that you're going to be transferring most of the data anyway, just do something like this: user@dest:/target$ nc -q 1 -l -p 1234 | tar xv ...


6

The disk quota is a limit on the disk space that a user can occupy. It has nothing to do with free space on disk. Look at the manual page for quota(1). So when trying to sync (write out data held in memory destined to files on disk) it finds that you aren't allowed to write that much data. And BTW, there is no "space in folders" in Unix/Linux. A directory ...


6

My question is, if I purchase a standard Windows external hard drive with a USB connection, will I be able to copy the files from the Linux cluster's files server to the external drive? Yes, there is no technical problem to this, however: The hardware us not a "standard windows hard drive with USB connection". Please scrap the windows part from that ...


6

So if I'm reading your question correctly you're doing: $ cp /nfs/mnt/foo.tar.gz /local/ext3/drive and the system crashes. I'd try isolating: $ cat /nfs/mnt/foo.tar.gz > /dev/null to check if it is the NFS system, and then $ dd if=/dev/zero of=/local/ext3/drive/zeros bs=1K count=8000 to check writes to the local file system. If both of those come ...


5

For account management, use LDAP. Simply install an LDAP client on all clients (e.g. the ldap-auth-client package on Ubuntu) and run an LDAP server on the server. Keep the home directories over NFS or Samba. The simplest setup is to mount the home filesystem as a whole on all clients at boot time. This doesn't provide good security because anyone who plugs ...


5

As Noufal Ibrahim says, I think this is a Solaris convention. IIRC, /export/home is used on the server where the actual files live, and /home is where the other servers mount it. What does mount | grep home say? I'm guessing that /export/home has a file system type of UFS, and /home has a type of NFS? /etc/fstab may also have some clues.


5

NFS itself is not generally considered secure - using the kerberos option as @matt suggests is one option, but your best bet if you have to use NFS is to use a secure VPN and run NFS over that - this way you at least protect the insecure filesystem from the Internet - ofcourse if someone breaches your VPN you're effectively wide open, but that would be the ...


5

I understand that mkdir is atomic, so perhaps: lockdir=/var/tmp/myapp if mkdir $lockdir; then # this is a new instance, store the pid echo $$ > $lockdir/PID else echo Job is already running, pid $(<$lockdir/PID) >&2 exit 6 fi # then set traps to cleanup upon script termination # ref http://www.shelldorado.com/goodcoding/tempfiles.html ...


5

I prefer to use hard links. lockfile=/var/lock/mylock tmpfile=${lockfile}.$$ echo $$ > $tmpfile if ln $tmpfile $lockfile 2>&-; then echo locked else echo locked by $(<$lockfile) rm $tmpfile exit fi trap "rm ${tmpfile} ${lockfile}" 0 1 2 3 15 # do what you need to Hard links are atomic over NFS and for the most part, mkdir is ...


5

The reasons are largely historical and pragmatic and date back to the technology that was incumbent in the 1980s and 1990s when much of the work on distributed systems architecture was being done: NFS is an open standard and is supported on pretty much every unix system built from the late 1980s onwards. Unix and NFS were the incumbent standard in the ...


5

Quoting verbatim from https://uisapp2.iu.edu/confluence-prd/pages/viewpage.action?pageId=123962105 : Under linux/unix, if you remove a file that a currently running process still has open, the file isn't really removed. Once the process closes the file, the OS then removes the file handle and frees up the disk blocks. This process is complicated ...


5

Normally when mounting NFS it's a good idea to have flags set similar to this: bg,intr,soft bg If the first NFS mount attempt times out, retry the mount in the background. After a mount operation is backgrounded, all subsequent mounts on the same NFS server will be backgrounded immediately, without first ...


5

An SSH server with SFTP service (which comes standard in most SSHD installs), it's also a platform independent solution. The setup is easy: sudo apt-get install ssh should fill in any of the server-side gaps you'd need. On your router, you'd want to map port 22 to the hosting machine, or you could reconfigure SSHD to listen on a different or additional ...


5

How? Or TL;DR The fastest method I've found is a combination of tar, mbuffer and ssh. E.g.: tar zcf - bigfile.m4p | mbuffer -s 1K -m 512 | ssh otherhost "tar zxf -" Using this I've achieved sustained local network transfers over 950 Mb/s on 1Gb links. Replace the paths in each tar command to be appropriate for what you're transferring. Why? mbuffer! ...


5

NFS doesn't have a concept of immutable files, which is why you get the error. I'd suggest that you just remove write access from everyone instead, which is probably close enough for your purposes. $ > foo $ chmod a-w foo $ echo bar > foo bash: foo: Permission denied The main differences between removing the write bit for all users instead of using ...


5

The reason you limit the number of inodes a user can access is so they don't make the system as a whole run out of inodes by creating a huge number of 0-byte files. With most Linux file systems (e.g. ext3 and ext4), each file (including device files) or directory has an inode -- a number used to point to a given file/directory. If a system runs out of ...


5

To run a program, it is opened by the kernel (or its interpreter). As long as it is running, the file is kept open. Deleting a file that is open in Unix deletes its name from the disk, the file itself is only deleted when it is closed. Add the above two, and you see that the running program continues to use its program file, and that one will be deleted ...


4

by default NFS is enabled you can remove packages: apt-get --purge remove nfs-kernel-server nfs-common portmap or stop services temporary: /etc/init.d/portmap stop /etc/init.d/nfs-kernel-server stop or stop them permanently: service portmap stop service nfs-kernel-server stop


4

offline caches are one thing, but what you're asking for is more difficult. If a file is modified both on the server and on the client while the two machines are not connected, someone has to decide which version to keep, or to merge the two versions. Requiring this kind of user input when the two machines reconnect doesn't fit well into the filesystem ...


4

You may be able to use Unison to synchronize your files. Unison uses the rsync protocol and can run over ssh. You may need to copy the executable into your directory on the remote system. Using rsync may cause problems as it is difficult to synchronize file deletions. EDIT: To sync a folder on A from system C (with working Drobox) a chosen directory ...



Only top voted, non community-wiki answers of a minimum length are eligible