Tag Info

Hot answers tagged

13

Your redirections have a race condition. This: >(wc -l | awk '{print $1}' > n.txt) runs in parallel with: awk 'BEGIN{getline n < "n.txt"}...' later in the pipeline. Sometimes, n.txt is still empty when the awk program starts running. This is (obliquely) documented in the Bash Reference Manual. In a pipeline: The output of each command in ...


13

Yes! You can use a process pipe |. ./preprocess.sh | while IFS= read -r line do ./research.sh "$line" & done A process pipe passes the standard output (stdout) of one process to the standard input (stdin) of the next. You can optionally put a newline character following a | and extend the command to the next line. Note: a|b is ...


6

You were really close: tr "o" "a" < <(echo "Foo") The substitution <() makes a file descriptor and just pastes the path to the shell. For comprehension just execute: <(echo blubb) You will see the error: -bash: /dev/fd/63: Permission denied That's why it just pastes /dev/fd/63 into the shell and /dev/fd/63 is not excutable, because it's ...


6

You can use bash process substitution: while IFS= read -r line do ./research.sh "$line" & done < <(./preprocess.sh) Some advantages of process substitution: No need to save temporary files. Better performance. Reading from another process often faster than writing to disk, then read back in. Save time to computation since when it is ...


5

Just write a subshell which sends to stdout and stderr... (echo STDOUT && echo STDERR >&2) For proof that it works: (echo STDOUT && echo STDERR >&2) > STDOUT.txt 2> STDERR.txt This will create files STDOUT.txt and STDERR.txt containing the words STDOUT and STDERR respectively.


5

pipe expression in process substitution causes a race condition in bash and ksh, zsh doesn't. The main problem here is that zsh waits, bash doesn't. You can see more details here. A quick fixed, adding sleep 1 in your awk to make n.txt always available: awk 'BEGIN{system("sleep 1");getline n < "n.txt"}{print $1 "\t" $1/n*100 "\t" $2}'


4

process_data() { awk -F /dev/fd/3 3<< \EOF awk code here EOF } Note that command line arguments can contain newline character, and while there's a length limit, it's general over a few hundred kilobyte. awk ' BEGIN {...} /.../ ... END {...} ' If the issue is about embedding single quote characters in the awk script, another approach is ...


4

A simple approach would be to use ls to list actual and imaginary files: ls . *.blah This assumes that there are visible files in the working directory and that you don't have any files that end in .blah1 1. ...and if you do, we won't judge you.


4

Yes, this is job for tee: rpm -qa | tee file | wc -l In this construction a | b a's stdout goes to stdin of b. In case of a > file | b all output form a goes to file and nothing goes to b stdin. tee command make a copy of all it receives on stdin to both file and stdout.


3

General, you can always do: <command> | ssh user@remote-server "cat > output.txt" It saves output of <command> to output.txt file in remote server. In your case, on Server-1: echo "qwerty" | ssh user@Server-2 "cat > output.txt" If two servers have no connectivity, but you can ssh to both servers, then from local machine, you can do: ...


3

I think your local shell is stripping off your quotes. You could try ssh remote sh -c '"echo hi > hi.txt"' When you send remote commands with ssh there are two shells involved with the reading of each line sent. Your local shell and your remote shell. A good explanation of this can be found at Unix/Linux Shell Quoting for remote shells


3

Yes, this is a job for tee: rpm -qa | tee file | wc -l Shell redirection (>) is just that — redirection — and you can only point the output stream to one other place at a time. There's nothing left for the | to see at that point. tee is made for just this purpose, where you want to split the stream into two parts, one going into a file and one still on ...


3

Why do you need to get the program from stdin? You could use single quotes ('), as Bash let's you split the contents between multiple lines. # awk 'BEGIN { sum = 0 } { sum += $1 } END { printf("sum = %d\n", sum) }' << EOF 1 2 3 EOF


3

So I just executed "< ./somefile.txt" in linux shell, Under bash, dash, and similar shells, that do not execute a command. That merely assigns somefile.txt to stdin. This goes nowhere because stdin is not used unless you supply a command. To use stdin for something try, for example, cat: cat <./somefile.txt Since cat echoes stdin to stdout, ...


3

This happens because you're only running the echo command as root. The output redirect is handled by your (non-root) shell. To avoid this, don't use the shell's redirect and use an actual command to handle the writing: tee. What you want to do can be done as so: echo "xyz" | sudo tee test > /dev/null (if you don't redirect the output, tee will output xyz ...


2

It depends on the shell. With zsh, this is described under Section "REDIRECTIONS WITH NO COMMAND". By default, READNULLCMD will be used as the command, which defaults to more.


2

The race condition is already identified. But if you'd like an easier solution, you don't need a separate wc to count the records, awk can do it: awk '{if($2!~/\*/){print $1;++n}END{print n >"n.txt"}' tmp | sort | uniq -c ... Beyond that, awk can count like sort|uniq -c as long as the values fit in memory, and also do the x/n calculation, but may ...


2

ls / /x df / /x wc / /etc/passwd od / /dev/null To guaranteed stdout written before stderr: (w;/) # Bourne/csh like shells only. sh -c 'w;/' 'time' w


2

This is probably the single most confusing, and annoying thing when working with ssh (at least in my opinion). The reason for this behavior is that ssh does not preserve arguments when executing a remote command. It takes all your arguments, and concatenates them together separated by spaces. So when you run ssh remote sh -c 'echo hi > hi.txt' In ...


1

I found the Bash One-Liners Explained series very useful in understanding more about all this stuff. Specifically the article linked above is all about input redirection. To solve the specific example above: > tr "o" "a" <<< $(echo "Foo") Faa


1

You're running into an output buffering problem. sed normally buffers its output when not writing to a terminal, so nothing gets written to the file until the buffer fills up (probably every 4K bytes). Use the -u option to sed to unbuffer output. clock -sf 'S%A, %B %d. %I:%M %P' | sed -u 's/\b0\+\([0-9]\+\)/\1/g' > testfile


1

I am so sorry, I just woke up. I was piping the output through grep and had my redirection on the wrong side of the pipe! I was redirecting the stderr of grep not yum! Mods, feel free to delete this, or leave it up if it has any value.


1

rpm -qa > file ; wc -l file should do what you want.


1

netcat springs to mind; it may be the more sensible choice (given the no-overhead, no compression approach to network communications) on your low-spec receiving machine. A nice usage example can be found here: http://stackoverflow.com/questions/4113986/example-of-using-named-pipes-in-linux-bash


1

You can write a function to use later: gen_stdout_stderr() { printf "%s\n" "STDERR" >&2 printf "%s\n" "STDOUT" } Then: $ gen_stdout_stderr STDERR STDOUT


1

I looked for sometime into top and there is no straight forward way to do this. As I mentioned earlier, you can use mpstat -P all > top.txt and then run your top command appending >> to output file for per user (you can use grep to filter... but that's a different topic :-). Can you elaborate what exactly the output you want to see (and is it part ...


1

If you know you're going to want to do this in advance ... Create a script session on your home computer (using -f to flush buffer): script -f output.txt rsync -vr /media/master /media/slave (Ctrl+D to finish the script session when get home) At work you can track output.txt: tail -f output.txt



Only top voted, non community-wiki answers of a minimum length are eligible