Hot answers tagged

148

Removing the first n lines (or bytes) can be done in-place using dd (or alternatively using loop devices). There will be no temp file and no size limit; however, it's dangerous since there will be no track of progress, and any error will leave you with a broken file. Create a sample file with 1000 lines: $ seq 1 1000 > 1000lines.txt $ head -n 3 1000lines....


137

I suggest the sed solution, but for the sake of completeness, awk 'NR >= 57890000 && NR <= 57890010' /path/to/file To cut out after the last line: awk 'NR < 57890000 { next } { print } NR == 57890010 { exit }' /path/to/file Speed test (here on macOS, YMMV on other systems): 100,000,000-line file generated by seq 100000000 > test.in ...


120

No, tail doesn't read the whole file, it seeks to the end then read blocks backwards until the expected number of lines have been reached, then it displays the lines in the proper direction until the end of the file, and possibly stays monitoring the file if the -f option is used. Note however that tail has no choice but to read the whole data if provided a ...


119

If you have enough space to compress the file, which should free a significant amount of space, allowing you to do other operations, you can try this: gzip file && zcat file.gz | tail -n +300000001 | gzip > newFile.gz That will first gzip the original input file (file) to create file.gz. Then, you zcat the newly created file.gz, pipe it through ...


103

Have you tried tail -f file1 file2? It appears to do exactly what you want, at least on my FreeBSD machine. Perhaps the tail that comes with a Debian system can do it too?


90

You can use tail command with -f : tail -f /var/log/syslog It's good solution for real time show.


82

You describe the GNU tail utility. The difference between these two flags is that if I open a file, a log file for example, like this: $ tail -f /var/log/messages ... and if the log rotation facility on my machine decides to rotate that log file while I'm watching messages being written to it ("rotate" means delete or move to another location etc.), the ...


73

You could use: journalctl -u service-name -f -f, --follow Show only the most recent journal entries, and continuously print new entries as they are appended to the journal. Here I've added "service-name" to distinguish this answer from others; you substitute the actual service name instead of the text service-name.


61

If you want lines X to Y inclusive (starting the numbering at 1), use tail -n "+$X" /path/to/file | head -n "$((Y-X+1))" tail will read and discard the first X-1 lines (there's no way around that), then read and print the following lines. head will read and print the requested number of lines, then exit. When head exits, tail receives a SIGPIPE signal and ...


61

Use the "watch" command: watch ls This will run the "ls" command every 2 seconds.


59

It's probably output buffering from grep. you can disable that with grep --line-buffered. But you don't need to pipe output from grep into awk. awk can do regexp pattern matching all by itself. tail -f test.txt | awk '/Beam/ {print $3}'


55

With GNU timeout: timeout 20 tail -f /path/to/file


52

tac only helps if you also use grep -m 1 (assuming GNU grep) to have grep stop after the first match: tac accounting.log | grep -m 1 foo From man grep: -m NUM, --max-count=NUM Stop reading a file after NUM matching lines. In the example in your question, both tac and grep need to process the entire file so using tac is kind of pointless. ...


50

Yes, less can follow by file name The feature has a fairly obscure syntax: less --follow-name +F file.log With less, --follow-name is different from the tail option --follow=name. It does not make less follow the file, instead it modifies the behaviour of the command key F inside of less to follow based on the file name, not the file descriptor. Also, ...


48

Use sed: sed -n -e '10,100p' input.txt > output.txt sed -n means don't print each line by default. -e means execute the next argument as a sed script. 10,100p is a sed script that means starting on line 10, until line 100 (inclusive), print (p) that line. Then the output is saved into output.txt. If your file is longer than suggested, this version (...


48

As said in the comments, Ctrl-C does not kill the tail process, which is done by sending either a SIGTERM or SIGKILL signal (the infamous -9...); it merely sends a SIGINT which tells tail to end the forward mode and exit. FYI, these's a better tool: less +F filename In less, you can press Ctrl-C to end forward mode and scroll through the file, then press ...


45

You can pipe the tail -f into sed, telling it to quit when it sees the line you're searching for: tail -f /path/to/file.log | sed '/^Finished: SUCCESS$/ q' sed will output each line it processes by default, and exit after it sees that line. The tail process will stop when it tries to write the next line and sees its output pipe is broken


43

I think you've covered the main point: less +F reads the whole file, whereas on many systems tail -f only reads the end of the file, and even on the systems where it does read the whole file, at least it doesn't keep the whole file in memory. That makes less +F impractical for very large files. You can, however, run less -n +F, which causes less to read only ...


39

Have a look at lnav, the advanced log file viewer. It can also pretty print various formats. Before: After:


39

Maybe: tail -n +1 -f file | awk '{printf "\r%lu", NR}' Beware that it would output a number for every line of input (though overriding the previous value if sent to a terminal). Or you can implement the tail -f by hand in shell: n=0 while :; do n=$(($n + $(wc -l))) printf '\r%s' "$n" sleep 1 done < file (note that it runs up to one wc and one ...


39

What about: tail -f file1 & tail -f file2 Or prefixing each line with the name of the file: tail -f file1 | sed 's/^/file1: /' & tail -f file2 | sed 's/^/file2: /' To follow all the files whose name match a pattern, you could implement the tail -f (which reads from the file every second continuously) with a zsh script like: #! /bin/zsh - ...


39

What you see is effect of a standard stdout buffer in grep provided by Glibc. The best solution is to disable it by using --line-buffered (GNU grep, I'm not sure what other implementations might support it or something similar). As for why this only happens in some cases: ssh server "tail -f /var/log/server.log | grep test" runs the whole command in the ...


38

There is a way better way of achieving this: less +F <file> It'll show you the whole file, has the full power of less and will wait for new input. If you want to stop waiting for input, and read a specific part, you can stop it with ^C and resume with F. The F command is always available in less, if you decide to watch for changes while having a ...


37

tail -f's tail is actually something unknown in the present, so how should the next tail know it. On the other hand tail -f's head is something already known and could therefor be processed. Or to put it simpler: tail is relative to the end of the file, but the output stream of tail -f got no EOF (at least not before its termination). If you find the first ...


37

It is possible like this, but as others have said, the safest option is the generation of a new file and then a move of that file to overwrite the original. The below method loads the lines into BASH, so depending on the number of lines from tail, that's going to affect the memory usage of the local shell to store the content of the log lines. The below ...


35

If you're getting this warning when using the StorNext filesystem and are running coreutils 8.21 or earlier, there isn't much to worry about; this warning message is expected. GNU tail has hardwired knowledge about a number of filesystem types, and warns when it encounters an unknown type. Support for the StorNext filesystem was added to tail in coreutils ...


35

You can try: tail -n0 -f x.log From man page: -n, --lines=K output the last K lines, instead of the last 10; or use -n +K to output lines starting with the Kth


35

If you're on Linux and the filesystem is ext4 or xfs, and the purpose of this move is to free up space on the file system (otherwise you can leave the file as is and just seek into it when reading it), you can use the collapse range feature of the fallocate system call or command line utility. The only problem is that it can only collapse full blocks, so you ...


33

You're looking for tail -f error.log (from man tail): -f, --follow[={name|descriptor}] output appended data as the file grows; -f, --follow, and --fol‐ low=descriptor are equivalent That will let you watch a file and see any changes made to it.


Only top voted, non community-wiki answers of a minimum length are eligible