Tag Info

Hot answers tagged

55

I ran this: strace -o spork.out bash -c "echo 1234 >> some-file" to figure out your question. This is what I found: open("some-file", O_WRONLY|O_CREAT|O_APPEND, 0666) = 3 No file named "some-file" existed in the directory in which I ran the echo command.


45

This is not only done in Bash, it's required by the standard. From the Single Unix Specification: Appended output redirection shall cause the file whose name results from the expansion of word to be opened for output on the designated file descriptor. The file is opened as if the open() function as defined in the System Interfaces volume of POSIX.1-2008 ...


27

Looking in the source, it does use O_APPEND. For bash 4.3.30 in make_cmd.c line 710-713 read: case r_appending_to: /* >>foo */ case r_append_err_and_out: /* &>> filename */ temp->flags = O_APPEND | O_WRONLY | O_CREAT; break;


26

Here's another way to do locking in shell script that can prevent the race condition you describe above, where two jobs may both pass line 3. The noclobber option will work in ksh and bash. Don't use set noclobber because you shouldn't be scripting in csh/tcsh. ;) lockfile=/var/tmp/mylock if ( set -o noclobber; echo "$$" > "$lockfile") 2> ...


17

Let's investigate that using strace on a local (non-NFS) filesystem: $ strace -eopen -- bash -c "echo foo >> /tmp/testfile000" 2>&1 | grep /tmp/testfile000 open("/tmp/testfile000", O_WRONLY|O_CREAT|O_APPEND, 0666) = 3 $ strace -eopen -- bash -c "echo foo > /tmp/testfile000" 2>&1 | grep /tmp/testfile000 open("/tmp/testfile000", ...


14

If you use NFSv4 with sec=krb5p, then it is secure. (That means use Kerberos 5 for authentication, and encrypt the connection for privacy.) But if you use NFS v3 or NFS v4 with sys=system, then no, it's not secure at all. There might also be some concern with exposing the kerberos and rpc ports to the internet at large, just in case of unknown ...


14

One way to delete files/direcories like this is by their inode-reference. To find the inodes for elements in current dir: ls -i 14813568 mikeaâcnt To delete this: find . -inum 14813568 -delete


12

unfs3 is dead as far as I know; Ganesha is the most active userspace NFS server project right now, though it is not completely mature. Although it serves different protocols, Samba is an example of a successful file server that operates in userspace. I haven't seen a recent performance comparison. Some other issues: Ordinary applications look files up ...


12

The quick way The quickest way to transfer files over a LAN is likely not rsync, unless there are few changes. rsync spends a fair bit of time doing checksums, calculating differences, etc. If you know that you're going to be transferring most of the data anyway, just do something like this: user@dest:/target$ nc -q 1 -l -p 1234 | tar xv ...


12

exportfs When you create a /etc/exports file on a server you need to make sure that you export it. Typically you'll want to run this command: $ exportfs -a This will export all the entries in the exports file. showmount The other thing I'll often do is from other machines I'll check any machine that's exporting NFS shares to the network using the ...


12

When you're looking for a file belonging to a package which is installed on your machine, you can use dpkg -S (equivalent to dpkg-query -S): dpkg -S /etc/exports In this case though it won't find anything, because /etc/exports is created by a maintainer script (and that type of file is explicitly not handled by dpkg-query, or for that matter by apt-file). ...


11

Normally when mounting NFS it's a good idea to have flags set similar to this: bg,intr,soft bg If the first NFS mount attempt times out, retry the mount in the background. After a mount operation is backgrounded, all subsequent mounts on the same NFS server will be backgrounded immediately, without first ...


11

Here are 2 ways to do it: mount Using mount's -v switch: $ mount -v | grep /home/sam mulder:/export/raid1/home/sam on /home/sam type nfs (rw,intr,tcp,nfsvers=3,rsize=16384,wsize=16384,addr=192.168.1.1) nfsstat Using nfsstat -m: $ nfsstat -m | grep -A 1 /home/sam /home/sam from mulder:/export/raid1/home/sam Flags: ...


10

NFS was designed with the idea that user and group ids would be the same on all machines across the network. For ordinary users, that works ok. But root's UID is always 0, and just because you have root on one box, it doesn't mean that you should have root access to every machine on the network. Therefore, NFS treats root specially. By default, root is ...


10

I'm pretty sure that you are looking at legacy concerns. Recall that the Perl5 manual was released in 1994 and that it was just an edit of Perl4's manual from 1991. In those days it could probably be said about the oft-named Nightmare File System that "it isn't how well the bear dances that amazes, but that it dances at all". NFS2 in the 1991 epoch was ...


10

Mount the NFS-share on the clients using the mount-options "bg,intr,hard". Most important in your case is "bg" for background - which tells the system not to block when the server is not available. "intr" for interrruptable - so you can kill hanging mounts on the client with the kill command. "hard" is the opposite of "soft". The difference is that "hard" ...


10

Olaf Kirch originally developed both the user space and kernel based version of the NFS server. In his year 2000 book, "Linux Network Administration" he says: The 2.2.0 kernel supports an experimental kernel-based NFS server developed by Olaf Kirch and further developed by H.J. Lu, G. Allan Morris, and Trond Myklebust. The kernel-based NFS support provides ...


9

NFS itself is not generally considered secure - using the kerberos option as @matt suggests is one option, but your best bet if you have to use NFS is to use a secure VPN and run NFS over that - this way you at least protect the insecure filesystem from the Internet - ofcourse if someone breaches your VPN you're effectively wide open, but that would be the ...


9

I prefer to use hard links. lockfile=/var/lock/mylock tmpfile=${lockfile}.$$ echo $$ > $tmpfile if ln $tmpfile $lockfile 2>&-; then echo locked else echo locked by $(<$lockfile) rm $tmpfile exit fi trap "rm ${tmpfile} ${lockfile}" 0 1 2 3 15 # do what you need to Hard links are atomic over NFS and for the most part, mkdir is ...


9

The reason you limit the number of inodes a user can access is so they don't make the system as a whole run out of inodes by creating a huge number of 0-byte files. With most Linux file systems (e.g. ext3 and ext4), each file (including device files) or directory has an inode -- a number used to point to a given file/directory. If a system runs out of ...


8

Remember that each of the NFS client systems will determine the username by looking up the numerical UID locally using the local system's /etc/passwd, or in your centralized user database. The NFS server only stores the UID in numerical format, and does not know about usernames. This is also true for group names vs. GIDs. In your case, serverA and serverB ...


8

The following excerpt from this essay potentially explains why that directory refuses to be deleted: NFSv4 requires that all filenames be exchanged using UTF-8 over the wire. The NFSv4 specification, RFC 3530, says that filenames should be UTF-8 encoded in section 1.4.3: “In a slight departure, file and directory names are encoded with UTF-8 to deal with ...


7

No. You can export a device file through NFS or some other network filesystems. But the meaning of the device file is dependent on the machine where you open it. If you export /dev/video0 over NFS from a server machine to a client machine, the client machine just sees “character device 81:0”, and interprets it as its own video capture device. The client ...


7

How? Or TL;DR The fastest method I've found is a combination of tar, mbuffer and ssh. E.g.: tar zcf - bigfile.m4p | mbuffer -s 1K -m 512 | ssh otherhost "tar zxf -" Using this I've achieved sustained local network transfers over 950 Mb/s on 1Gb links. Replace the paths in each tar command to be appropriate for what you're transferring. Why? mbuffer! ...


7

The disk quota is a limit on the disk space that a user can occupy. It has nothing to do with free space on disk. Look at the manual page for quota(1). So when trying to sync (write out data held in memory destined to files on disk) it finds that you aren't allowed to write that much data. And BTW, there is no "space in folders" in Unix/Linux. A directory ...


7

You need to run the command on the server after modifying the /etc/exports file: $ exportfs -a Also when debugging connectivity issues with NFS you can run the command showmount -e <nfs server> to see what mounts a given server is exporting out. example $ showmount -e cobbler Export list for cobbler: /cobbler/isos 192.168.1.0/24 services running ...


7

Use nfsstat -m it will display all the nfs mounted filesystem and theirs properties.


7

You should not use non-ASCII characters in the command line since as you could see, for some reason, they won't necessarily correspond to the filename (Unicode has various ways for expressing accented letters). Something like: rm -rf mike* should work since the filename is directly generated by the shell. But make sure there's only one match (do an echo ...


6

Bad form, I know, to answer my own question, but.... I needed a couple more steps, outlined here. In short, I needed to execute: sudo nfsd update As another detail, I added the client name to the export and removed the "-rw" flag.


6

by default NFS is enabled you can remove packages: apt-get --purge remove nfs-kernel-server nfs-common portmap or stop services temporary: /etc/init.d/portmap stop /etc/init.d/nfs-kernel-server stop or stop them permanently: service portmap stop service nfs-kernel-server stop



Only top voted, non community-wiki answers of a minimum length are eligible