Take the 2-minute tour ×
Unix & Linux Stack Exchange is a question and answer site for users of Linux, FreeBSD and other Un*x-like operating systems.. It's 100% free, no registration required.

According to the bash man page:

The redirection operator

   [n]<&digit-

moves the file descriptor digit to file descriptor n, or the standard input (file descriptor 0) if n is not specified. digit is closed after being dupli‐cated to n.

What does it mean to "move" a file descriptor to another one? What are the typical situations for such practice?

share|improve this question
add comment

3 Answers

up vote 6 down vote accepted

3>&4- is a ksh93 extension also supported by bash and that is short for 3>&4 4>&-, that is 3 now points to where 4 used to, and 4 is now closed, so what was pointed to by 4 has now moved to 3.

Typical usage would be in cases where you've duplicated stdin or stdout to save a copy of it and want to restore it, like in:

Suppose you want to capture the stderr of a command (and stderr only) while leaving stdout alone in a variable.

Command substitution var=$(cmd), creates a pipe. The writing end of the pipe becomes cmd's stdout (file descriptor 1) and the other end is read by the shell to fill up the variable.

Now, if you want stderr to go to the variable, you could do: var=$(cmd 2>&1). Now both fd 1 (stdout) and 2 (stderr) go to the pipe (and eventually to the variable), which is only half of what we want.

If we do var=$(cmd 2>&1-) (short for var=$(cmd 2>&1 >&-), now only cmd's stderr goes to the pipe, but fd 1 is closed. If cmd tries to write any output, that would return with a EBADF error, if it opens a file, it will get the first free fd and the open file will be assigned it to stdout unless the command guards against that! Not what we want either.

If we want the stdout of cmd to be left alone, that is to point to the same resource that it pointed to outside the command substitution, then we need somehow to bring that resource inside the command substitution. For that we can do a copy of stdout outside the command substitution to take it inside.

{
  var=$(cmd)
} 3>&1

Which is a cleaner way to write:

exec 3>&1
var=$(cmd)
exec 3>&-

(which also has the benefit of restoring fd 3 instead of closing it in the end).

Then upon the { (or the exec 3>&1) and up to the }, both fd 1 and 3 point to the same resource fd 1 pointed to initially. fd 3 will also point to that resource inside the command substitution (command substitution only redirects the fd 1, stdout). So above, for cmd, we've got for fds 1, 2, 3:

  1. the pipe to var
  2. untouched
  3. same as what 1 points to outside the command substitution

If we change it to:

{
  var=$(cmd 2>&1 >&3)
} 3>&1-

Then it becomes:

  1. same as what 1 points to outside the command substitution
  2. the pipe to var
  3. same as what 1 points to outside the command substitution

Now, we've got what we wanted: stderr goes to the pipe and stdout is left untouched. However, we're leaking that fd 3 to cmd.

While commands (by convention) assume fds 0 to 2 to be open and be standard input, output and error, they don't assume anything of other fds. Most likely they will leave that fd 3 untouched. If they need another file descriptor, they'll just do an open()/dup()/socket()... which will return the first available file descriptor. If (like a shell script that does exec 3>&1) they need to use that fd specifically, they will first assign it to something (and in that process, the resource held by our fd 3 will be released by that process).

It's good practice to close that fd 3 since cmd doesn't make use of it, but it's no big deal if we leave it assigned before we call cmd. The problems may be: that cmd (and potentially other processes that it spawns) has one fewer fd available to it. A potentially more serious problem is if the resource that that fd points to may end up held by a process spawned by that cmd in background. It can be a concern if that resource is a pipe or other inter-process communication channel (like when your script is being run as script_output=$(your-script)), as that will mean the process reading from the other end will never see end-of-file until that background process terminates.

So here, it's better to write:

{
  var=$(cmd 2>&1 >&3 3>&-)
} 3>&1

Which, with bash can be shorten to:

{
  var=$(cmd 2>&1 >&3-)
} 3>&1

To sum up the reasons why it's rarely used:

  1. it's non-standard and just syntactic sugar. You've got to balance saving a few keystrokes with making your script less portable and less obvious to people not used to that uncommon feature.
  2. The need to close the original fd after duplicating it is often overlooked because most of the time, we don't suffer from the consequence, so we just to >&3 instead of >&3- or >&3 3>&-.

Proof that it's rarely used, as you found out is that it is bogus in bash. In bash compound-command 3>&4- or any-builtin 3>&4- leaves fd 4 closed even after compound-command or any-builtin has returned. A patch to fix the issue is now (2013-02-19) available.

share|improve this answer
    
Thanks, now I know what moving a fd is. I have 4 basic questions regarding the 2nd snippet (and fds in general): 1) In cmd1, you make 2 (stderr) to be a copy of 3, what if the command would internally use this 3 fd? 2) Why do 3>&1 and 4>&1 work? Duplicating 3 and 4 takes effect only in those two cmds, is the current shell also affected? 3) Why do you close 4 in cmd1 and 3 in cmd2? Those commands don't use mentioned fds, do they? 4) In the last snippet, what happens if a fd is duplicated to a non-existent one (in cmd1 and cmd2), I mean to 3 and 4, respectively? –  Quentin Feb 17 '13 at 2:25
    
@Quentin, I've used a simpler example and explained a bit more hoping it now raises fewer questions than it answers. If you've still got questions not directly related to the fd moving syntax, I suggest you ask a separate question –  Stephane Chazelas Feb 17 '13 at 17:49
    
{ var=$(cmd 2>&1 >&3) ; } 3>&1- Isn't it a typo in closing 1? –  Quentin Feb 18 '13 at 15:50
    
@Quentin, It's a typo in that I don't think I intended to include it, however it makes no difference since nothing inside the braces uses that fd 1 (it was the whole point of duplicating it to fd 3: because the original 1 otherwise wouldn't be accessible inside $(...)). –  Stephane Chazelas Feb 18 '13 at 16:14
1  
@Quentin Upon entering {...}, fd 3 points to what fd 1 used to point and fd 1 is closed, then upon entering $(...), fd 1 is set to the pipe that feeds $var, then for cmd 2 to that as well, and then 1 to what 3 points to, that is the outer 1. The fact that 1 remains closed afterwards is a bug in bash, I'll report it. ksh93 where that feature comes from doesn't have that bug. –  Stephane Chazelas Feb 18 '13 at 18:30
show 1 more comment

It means to make it point to the same place the other file descriptor does. You need to do this very rarely, apart from the obvious separate handling of the standard error descriptor (stderr, fd 2, /dev/stderr -> /proc/self/fd/2). It can come handy in some complex cases.

The Advanced Bash Scripting guide has this longer log level example and this snippet:

# Redirecting only stderr to a pipe.
exec 3>&1                              # Save current "value" of stdout.
ls -l 2>&1 >&3 3>&- | grep bad 3>&-    # Close fd 3 for 'grep' (but not 'ls').
#              ^^^^   ^^^^
exec 3>&-                              # Now close it for the remainder of the script.

In Source Mage's Sorcery we for example use it to discern different outputs from the same code block:

  (
    # everything is set, so run the actual build infrastructure
    run_build
  ) 3> >(tee -a $C_LOG >> /dev/stdout) \
    2> >(tee -a $C_LOG 1>&2 > $VOYEUR_STDERR) \
     > >(tee -a $C_LOG > $VOYEUR_STDOUT)

It has extra process substitution appended for logging reasons (VOYEUR decides whether the data should be shown on screen or just log), but some messages need to always be presented. To achieve that, we print them to file descriptor 3 and then handle it specially.

share|improve this answer
add comment

In Unix, files are handled by file descriptors (smallish integers, for example standard input is 0, standard output is 1, standard error is 2; as you open other files they normally get assigned the smallest unused descriptor). So if you know the inards of the program, and you want to send the output that goes to file descriptor 5 to the standard output, you'd move descriptor 5 to 1. That's where the 2> errors comes from, and constructions like 2>&1 to duplicate errors into the output stream.

So, hardly ever used (I vaguely remember using it once or twice in anger in my 25+ years of almost exclusive Unix use), but when needed absolutely essential.

share|improve this answer
    
But why not to duplicate file descriptor 1 in the following way: 5>&1? I don't understand what's the use of moving FD since the kernel is about to close it right after... –  Quentin Feb 16 '13 at 18:57
    
That doesn't duplicate 5 into 1, it sends 5 to where 1 is going. And before you ask; yes, there are several different notations, picked up from a variety of precursor shells. –  vonbrand Feb 16 '13 at 19:01
    
Still don't get it. If 5>&1 sends 5 to where 1 is going, then what exactly does 1>&5- do, besides closing 5? –  Quentin Feb 16 '13 at 19:23
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.