Questions tagged [tail]
Watching updates at the end of a file, with or without the tail utility
412
questions
0
votes
2
answers
100
views
How to view a log file that's worth 10GB+?
The ways that I've thought of:
If you get to reproduce the scenario in real time. Use
tail -f application.log | tee /tmp/tailed_log
But it's not sure that we'll get to reproduce the scenario in real ...
- 434
0
votes
2
answers
51
views
Side by side output from two `tail -f` commands
I have two scripts emitting single string to log files periodically that I need to tail -f and combine side by side for which I am using paste. This works but I am unable to pipe the output to another ...
- 9,086
3
votes
1
answer
126
views
Save the output of earlier tailed logs to a file?
Is it possible to save the output of earlier tailed logs to a file?
Steps to Reproduce this issue:
tail -f application.log
Ctrl+C to end the logs
Save that output to a file now.
I am assuming an ...
- 434
0
votes
1
answer
67
views
Continuously display last line of command output without clearing the screen
I want to print only the last line of the output of a command on the same line continuously without clearing the screen. This would be part of another function. Let's say my script has the following ...
- 1
1
vote
1
answer
269
views
How to stop "tail -f" in a script and exit if a certain condition is fulfilled?
I'm trying to come up with a script for managing jobs on a supercomputer. The details don't matter much, but a key point is that the script starts to tail -f a file once it appears. Now this would run ...
- 286
0
votes
1
answer
46
views
Sleep vs tail to wait on process
Currently I do:
while [ -d "/proc/$PID" ]; do
sleep 1
done
To wait for a process to exit. If I would replace it with:
tail --pid=$PID -f /dev/null
Would that be more efficient for the ...
- 179
1
vote
1
answer
71
views
How to make tail command end itself
In my app I use tail -q -f -n +1 [file name] to read current data of a file (dynamically growing file) and continue reading new chunks until the end. But that command never exits. It stays alive in ...
- 31
2
votes
6
answers
165
views
Only pipe output if at least n lines
I often want to grep output from processes that include a few header lines. But if grep would remove all actual content lines, I don't want to display the header lines. For this to work I would need a ...
- 254
1
vote
0
answers
55
views
Random timeout on basic operation in bash
Context
I have a GitHub Action running on a linux machine.
It does the following (might want to skip to problem first as most of this might be irrelevant):
pwd: /home/runner/work/net.twisterrob....
- 331
2
votes
4
answers
799
views
Grep and find to get the last match in multiple files [duplicate]
Assume two files with the following content:
$ cat ttest1.txt
x = 1
x = 3
y = 5
$ cat ttest2.txt
x = 4
x = 10
y = 3
I would like to recursively grep the files for x and print the last instance in ...
- 1,151
0
votes
1
answer
43
views
maintain appended logs in separate file despite tail command termination
I have tomcat catalina.log file that keeps getting updated.
I wish to maintain its past 500 lines + any appended logs into a new file catalina.log_new for the next 2 hours.
To start off I run nohup ...
- 459
0
votes
1
answer
91
views
How to exit a FOR loop on detecton of "error" stream using TAIL
I have a "for" loop which passes the results to a file called "results"
Occasionally an error stream is detected and the loop continues trying within the loop for a number of times ...
- 17
0
votes
1
answer
111
views
How to NOT append nohup output to a file but replace it?
Lately I needed to be able to run a script in the background and monitor its output from another machine. For this reason I was able to use this command:
$ nohup ./rsync.sh > ./output.txt
This is ...
- 1,181
12
votes
2
answers
2k
views
Why can't I do two greps after a tail?
I'm able to run this command successfully:
tail -f my_file.txt | grep foo
It shows only the lines with the string foo, and it keeps showing them.
But when I run this command:
tail -f my_file.txt | ...
- 1,795
2
votes
1
answer
70
views
Flush output of `tail -f` into a file?
I have an app which dumps a lot of log messages. I want to catch a middle of that log, which happens as a reaction to an event.
Tried this:
tmpfile=`mktemp`
tail -n1 -f appA_log_file.txt >> $...
- 4,519
1
vote
1
answer
75
views
"tail -f", but on a file which gets rewritten (downloaded) again and again without outputting then content over and over again?
I've got log files which get downloaded via cron job. If a file is updated on the remote location, the local copy gets rewritten from the beginning even if only data has been appended.
Tools like tail ...
- 285
2
votes
1
answer
185
views
tail -1000f doesn't show the same output as tail -1000
I am trying to execute
tail -100000f MXMLExchangeMonitoring_Mx3.log | egrep "INTENTO.*Workflow" | cut -d ":" -f "3" | cut -c 3,4,5,6 | awk '{if($1>20)print$1}'
and I'...
0
votes
1
answer
49
views
How to follow logs while avoiding line wrapping w/ horizontal scroll enabled?
In the terminal, if I don't want to have logs of a program wrap, I can pipe those to less -S.
However, horizontal scroll with keyboard doesn't work unless I Ctrl+C, and after that, I don't know how to ...
- 101
0
votes
0
answers
60
views
Watch file for specific text and exit
I'm trying to write a command that exits cleanly when a specific string is eventually written to a log file.
I've tried the following but it never exits.
Setup: create a file for the purposes of ...
- 101
1
vote
0
answers
21
views
Weird grep behavior with invalid UTF8 input [duplicate]
I need to monitor a log file which sometimes contains invalid UTF8 characters.
Like this:
shaozr@fedora utf8-cut (bear_from_3.1.18) $ tail -F a.txt
a我々はMozartが好きです。
a我々はBachも好きです。
a▒▒▒々はMozartが好きです。
a▒...
- 215
1
vote
1
answer
182
views
How is the default stdout buffering set?
Running on Red Hat EL7, we have these great long lines in our log files so I do a
tail -f Log | cut -c1-$COLUMNS
This works great on some systems but other--apparently identical--systems, the pipe ...
- 301
3
votes
3
answers
236
views
How to get around issue with `tail -f` not emitting EOF and make it works with tools like `csvtool`?
I basically want to do this:
tail -f trades.csv | csvtool readable -
I want to read a CSV file in a readable format using csvtool and I want to keep watching it.
I think that command doesn't work ...
- 155
2
votes
3
answers
987
views
Shell script to filter string from a data stream and output to screen
I am trying to make it easy to tail a log file and print to the screen only the output of a single field.
I have the log file that looks like this on a Linux machine:
2022-10-21 16:00:08;areq_in=0;...
- 45
1
vote
0
answers
13
views
What's wrong with my log file generation?
I'd like to capture and filter the logs from my ESP32 peripheral from the serial port and generate a new log file with the selected line.
I do as follows:
$ (((stty speed 115200 && cat) </...
- 163
0
votes
2
answers
98
views
Select a microservice with awk and create a file with the current date and version
I would like to see the logs for a microservice with the latest version and send its content to a file according to the date.
user@MacBook-Pro ~ % kubectl -n bci-api get pods | grep ms-example-...
- 365
1
vote
2
answers
50
views
How to terminate _follow_ mode in **less** (`+F`) from pipe?
Assuming the following dummy command in bash:
### dummy long_operation_cmd function, for easy reproduction:
function long_operation_cmd() {
echo "operation 1"
sleep 5
echo "...
- 638
1
vote
1
answer
93
views
bash looping over files while tailing logs
I'm running a daemon that process files in a specific dir.
I want to process those one by one. so idea is I will copy files 1 by 1 in daemon dir, and tail its log, 2nd file should be copied to daemon ...
- 1,889
1
vote
3
answers
774
views
How to suffix or prefix each line coming from tail command
I have a file and I am running a tail command on this file like this
tail -n+1 -F "./log/catalina.log"
I want to suffix or prefix another string to each line coming out of this file, which ...
0
votes
0
answers
166
views
SCP results look strange when using >> to append to file
I am trying to record a reoccurring test to a txt file using tail but there is an issue with the output.
The commands I am running are:
script -q /dev/stdout -c 'scp /home/user/test.img user@pc28:/tmp'...
- 1
0
votes
1
answer
486
views
Can I pipe live data from the \watch command of postgresql into another command?
I'd like to pipe live data from psql into another command for purposes of monitoring via the shell.
Can this be done?
So far I have the following:
$ psql bareos -c '\watch (SELECT * FROM log ORDER BY ...
- 4,471
0
votes
0
answers
180
views
tail -f executed of ssh repeat output multiple times
I am trying to monitor a file remotely. Since I cannot install additional useful applications on the source I plan to use tail over ssh to bring the information local where I have more options.
Since ...
3
votes
2
answers
369
views
piping head and tail with globbing
I have several files in the same dir for which I want to print certain lines from a continuous interval, e.g., from 15th to 20th.
For a single file, this works head -n20 file.txt | tail -n6,
but how I ...
- 191
2
votes
2
answers
1k
views
Get the nth last line from file
I have a file with quite many lines, and I would need the nth last line (specifically the 95th from bottom). How could I go on about doing this? One way would be to use tail but then it prints ...
- 23
0
votes
1
answer
33
views
How to add empty new lines in less (log viewer)?
When I monitor some log file with tail -F file.log, I can press enter to add some space between actual log lines and the new ones, for example:
some log 1
some log 2
some log 3
some log 4
some log 5
...
- 761
0
votes
2
answers
257
views
how to invoke sed from awk while following logfile?
I'm supposed to follow a logfile, I want to initiate a sed command to edit conffile upon appearance of certain line in the log. I did little research and found out that it can be done with awk.
and ...
- 1,889
0
votes
0
answers
1k
views
Tail and watch, filter and extract string [duplicate]
Lets says I have a file called test.log more rows being added all the time:
one|two|apple|four|foo1
one|two|pear|four|foo2
one|two|apple|four|foo3
one|two|peach|four|foo4
I want to tail it but am ...
- 111
2
votes
1
answer
234
views
No output: busybox find . -exec sh -c ' readlink -f "$1" | tail -n +2 ' sh {} \;
I got access to only Busybox 1.31.1
I originally wanted to remove the current working directory of my output (the single dot).
Example:
/prueba$ ls
uno dos tres
When I:
$ busybox find .
.
./uno
./dos
...
- 143
1
vote
1
answer
218
views
Unix Script - On specific pattern, need to exit tail -f, and use awk to run separate script
So, I need to make a modification to a start script to clean up the /tmp once the application is fully loaded. After some research, I found a command for my script that does this:
tail -f /home/user1/...
- 133
0
votes
0
answers
207
views
script in bash two if condition and print in for loop
I'm learning to script in bash, trying to solve a little exercise. The exercise is this:
If the variable named "basenew" contains the contents of the variable
named "valuebase". &...
- 11
2
votes
1
answer
302
views
less in "tail -f mode" and ability to scroll horizontally at the same time
less -S is useful to have horizontal scrolling with arrow keys (like mentioned in Horizontal scrolling in smaller increments with less -S) instead of breaking long lines into multiple lines
less +F, ...
- 2,351
2
votes
1
answer
346
views
How to properly use tail to concatenate all hidden files [duplicate]
Issue
I want to be able to :
concatenate all files in a directory (regular and hidden),
but I would also like to display the title of each file at the beginning of each concatenation.
I found some ...
- 135
1
vote
2
answers
223
views
Efficient header replacement for large (10M line) files?
I am trying to rename the column headers of a large file, and I want to know the most efficient way to do so. Files are in the range of 10M to 50M lines, with ~100 characters per line in 10 columns.
A ...
- 111
1
vote
1
answer
135
views
how to logresolve httpd log file to /dev/null and tail it?
I would like to watch traffic live (httpd log activity) ,
I currently use tail -f /var/www/access.log
It works fine , its live, however just want to convert ip address to hostnames, i dont want to ...
- 201
0
votes
1
answer
721
views
Why isn't tail command tracking the changes in a file in realtime? [duplicate]
When I track a file in realtime using descriptor i.e tail --follow=descriptor file_name, it doesn't show the changes made in the file, whereas if i track the file in realtime using name i.e tail --...
- 167
1
vote
5
answers
398
views
Pipe skip 99 lines out of every 100
I have a bash commands pipeline that produces a ton of logging text output. But mostly it repeats the previous line except for the timestamp and some minor flags, the main output data changes only ...
- 121
1
vote
1
answer
1k
views
tail -1 file1 >> file2 without \n
it might be simple but after looking around I could not find a way to resolve my issue,
I am recovering the last line from file1 using tail -1 file1 and want to add it to an existing file that is a ...
- 13
0
votes
1
answer
796
views
Bash - "timeout" with tail issue
timeout 0.6s $(tail -f ./file.txt | grep -qA2 --line-buffered "detail: " | grep -E -- "$KeyNumbers")
So timeout isn't functioning as intended with this. I want it to cancel if ...
- 3
0
votes
1
answer
553
views
"tail -f | grep -q" as a trigger without worrying about the previous found
Currently, the only way to see if my Tomcat has fully started up is to check apache-tomcat log:
tail -f ./catalina.out
and look for this pattern to show up before I do any job.
org.apache.catalina....
- 385
2
votes
2
answers
174
views
Live tail from different folders with inclusion and exclusion of files
I want to live tail from three different folders with certain files included & excluded.
Folder 1:
Path: /var/www/app/var/log/
Exclude only: file1.log & file2.log
Folder 2:
Path: /var/log/...
- 485
0
votes
0
answers
170
views
why SIGPIPE behaves different in different kernels?
I have two different systems running with linux and I'm getting some logs with
tail -f logfile
While tail was running, I sent SIGPIPE to tail's process. tail continued to run on system 1 but died ...
- 1