Questions tagged [filter]
The filter tag has no usage guidance.
192
questions
1
vote
0
answers
16
views
How to set bandwidth limit using linux tc
In my linux router:
interface eth1 total bandwidth is 1gbit
I want to divide 1140kbit to GroupA, divide 150kbit to GroupB
Set users 10.10.10.158, 10.10.21.5, 10.10.21.6 to GroupB
Each user has no ...
- 11
0
votes
1
answer
16
views
Filter duplicated based on values of another column
I have the following example of a dataframe.Where you see that elements of 3nd column could be duplicated.I want to keep the entry which has the highest value in column 5
Meaning that for AGCCCGGGG I ...
0
votes
1
answer
18
views
Find entries that match 2 criteria, one based on a column one based on order
I have a tab-delimited file look like this. First column is read name and third is a column specifying if the read is Bacteria or Eukaryota. Each read could have many entries/hits. I want to extract ...
0
votes
1
answer
109
views
Rsync with date filter over ssh
I use the following command successfully
rsync -e 'ssh' -avr [email protected]:/home/mikrotik /bck/mikrotik/
How can I add date filter to this command? I would like to sync only files that are newer ...
- 103
2
votes
1
answer
60
views
How can I re{duce,move} the Angry Fruit Salad effect [duplicate]
Above, you can see the output of the kscreen-doctor -o command on Suse Tumbleweed.
I tried kscreen-doctor -o | cat to no avail.
Short of using a black background as a damage limiting device (still ...
- 1,222
1
vote
1
answer
89
views
checksum a pipeline
I want to construct a huge blob of data (a backup of a sort) and send it over the network (ssh or rsync) to another host. There is enough space on the remote for the data, but not on the local host, ...
- 518
0
votes
0
answers
175
views
How can I use rsync filters properly?
I'm having trouble in getting rsync to respect my .rsync-filter files; which, I guess, it is due to me not properly understanding how they work.
Basically, I have made a filter inside my $HOME folder ...
- 1,835
1
vote
1
answer
186
views
How to use variables (that change in loop) in awk statement to cut a file
I have a large file that needs to be filtered by the first field (which is never repeated). Example as below:
NC_056429.1_398 2 3 0.333333 0.333333 0.333333 0.941178
NC_056429.1_1199 2 ...
- 13
1
vote
1
answer
60
views
Filtering lines of two files based on a third list file
I have 3 files: list_file, file1, and file2. I want to extract entire lines from file1 and file2 based on a list_file pairwise and concatenate the results on output.
Namely, I need to extract only ...
- 13
0
votes
1
answer
60
views
grep unique results but show full line containing the match
I'd like to filter an input like this
foo 2022-11-11
foo 2022-12-11
something else
bar 2022-12-07
to obtain
foo 2022-11-11
bar 2022-12-07
I'm starting with grep -oP "^[A-z]{3}" | sort -u ...
- 105
0
votes
0
answers
30
views
How can Postfix smtp server scan or calculate the md5sum of an attachment while it enters to my server?
How can a Postfix SMTP server scan or calculate the md5sum of an attachment which arrived to my server?
The issue comes from a recurring attachment that is sent with different body and different email ...
- 131
0
votes
1
answer
170
views
Passing value with xargs to generate dynamic output filename
I have some files in a folder. By running the following command I can create a list with the filtered filenames ( test1, test2,test3) and pass it using xargs to a grep filtering command filter file in ...
- 49
1
vote
0
answers
140
views
postfix: conditionally ignoring certain "User unknown in virtual mailbox table" errors when using a milter
NOTE: I have voted to close this question here, and I'm moving it to ServerFault, where there seem to be more questions and answers involving postfix.
I'm running postfix version 2.93 under Debian 8, ...
- 535
0
votes
1
answer
47
views
Remove/move files in a directory with filename timestamp pattern
So I have files that need to remove/move/filter.
All files have this pattern like this in a directory, let's say directory frames contain this pattern filename timestamp_in_nanosecond.jpg.
And this is ...
3
votes
1
answer
468
views
How does tmux list-panes -f (filter) work?
I read the man page but I could not find how to use -f when using tmux list-panes. It seems to be a filter, I assume this works like grep in some way?
Does anyone know how -f works?
- 542
0
votes
1
answer
611
views
How to filter rows by one column?
Sorry if this is a basic question, I don't use linux often but I have a 13GB file that I want to filter down.
I have 123 columns and I want to remove rows that have only a . in the 75th column.
I've ...
- 3
4
votes
1
answer
410
views
Find directories with name, but filter them if they contain a specific top-level file in them
I want to find all directories that are named Contents and then from the list of found results, I want to filter them and only include those that have a file named Database.json directly inside them.
...
- 545
1
vote
0
answers
66
views
How to import Kmail filters into Evolution?
Is there a way to import filters defined in Kmail into Evolution? I know how to export them (Tools → Import/export → Kmail/config), and extract the filters (extract config/filters which describes them ...
- 321
0
votes
1
answer
675
views
How to combine find and cp in linux to copy specific files
I am having the following problem in linux:
I need to find specific files from a directory and copy those files (if they exist) into another directory
this is the command (i need to find specific text ...
- 3
2
votes
2
answers
389
views
Filtering file list to show only files alphabetically after certain file
I have a huge number of images from which I'd like to feed some to ffmpeg time to time.
But I only want to feed ones that are alphabetically after certain image (last frame of previous run, name ...
- 23
-1
votes
3
answers
58
views
How to strip characters by argument?
I want to filter any comma and any double quote mark from output by some command with.
For some entries.
Pseudocode:
removechar --any -, -"
Current output could look like any of these
lorem, ...
- 113
1
vote
1
answer
780
views
Filtering a file based on values from another file
I have two files
file1:
U 20 100 1_A 1_A
U 14 200 1_B 1_B
U 14 300 1_C 1_C
file2:
D 12 90 1_A 1_A
D 15 97 1_A 1_A
D ...
- 59
0
votes
1
answer
200
views
How Do I Get `tree` to Selectively Exclude Contents of Directories?
I'm using tree on a directory which has -- among other things -- some Audacity projects in it. Each Audacity project is some file X.aup, along side a directory X_data, which contains a few sub-...
- 11
0
votes
1
answer
502
views
Stream filtering changing log file in realtime
I have file.log with this sample:
11-Feb-2022 18:34:42 lorem uneeded5
11-Feb-2022 18:34:43 water uneeded6
11-Feb-2022 18:34:44 ipsum uneeded7
11-Feb-2022 18.34.45 waterfall uneeded8
I expect output ...
-1
votes
3
answers
114
views
Trim pathname in CSV file
I am struggling over a filter where I am trying to trim data in a specific column of a CSV after the 3rd or nth occurrence of the character \.
My data looks something like this:
data,data,c:\path1\...
- 11
0
votes
1
answer
424
views
date string column to timestamp column in a log file
I have log.txt with sample output like these:
...
10-Feb-2022 15:15:14.099 lorem
10-Feb-2022 15:15:15.133 ipsum
10-Feb-2022 15:15:16.233 dolor
...
I expect the output of filtered log.txt will be
...
...
0
votes
1
answer
319
views
Filter changing file periodically and redirect filtered output to new file
Suppose I have log.txt
The sample of log.txt are like these format:
Code Data Timestamp
...
C:57 hello 1644498429
C:56 world 1644498430
C:57 water 1644498433
...
If I want filter string line that ...
0
votes
0
answers
2k
views
Copying files from remote to local using lftp in unix
I originally posted this in server fault, but it didn't get much traction. I'm hoping someone here can help.
I'm trying to write a script to copy some files from a remote ftp server to a local ...
- 51
1
vote
3
answers
82
views
How to get the list all files that are x.y but are not z/x.y?
I have a lot of Settings.json files on this machine.
I get the list of all of them using:
find / -name "Settings.json" 2>&-
But some of them are in directories that end to Api. For ...
- 545
0
votes
1
answer
4k
views
nftables refuses to add rules "Error: Could not process rule: No such file or directory"
Problem:
When trying to add nft rules, I keep getting "Error: Could not process rule: No such file or directory"
Example command:
nft add rule netdev filter input ip daddr 10.10.10.10 ...
- 542
1
vote
1
answer
624
views
Linux not responding to public ip ssh request
I'm trying to connect via ssh to my home server from outside of my home network, but I don't get any respond.
When I try to locally ssh that server, everything is just fine.
But when I do it from ...
- 11
1
vote
1
answer
268
views
grep -v does not return complement of grep
I want to filter a file for lines starting with a space. I use the following command:
grep -v "^ " < input > input_no_starting_space
To double check my results, I run the following:
...
- 113
1
vote
1
answer
60
views
Filter phots into directories by date range
I have set up our family mobile phones to automatically backup all photos taken to a NAS (Qnap TS-231P), one folder per mobile. For one phone, for some reason a lot of photos were copied multiple ...
- 11
0
votes
0
answers
614
views
Illegal "match" error when applying Linux tc, u32 filters mask
I want apply the following rules for TOS based traffic shaping. All rules works but the final rule which mask the IP header shows illegal "match" error
sudo tc qdisc del dev tun0 root handle ...
1
vote
1
answer
1k
views
Defining functions and using map in jq
Can you explain to me how the following two differ?
jq ´def addvalue(f): . + [f]; map(addvalue(.[0]))´
[[1,2],[10,20]]
=> [[1,2,1], [10,20,10]]
jq ´def addvalue(f): f as $x | map(. + $x); ...
- 249
0
votes
0
answers
288
views
IPTABLES Limit connection per ip
I have ubuntu 20 server with nginx, php, etc
How i can block ips (http port 80, 443) where connections > 20, but don't block proxy ip 1.2.3.4.5
0
votes
0
answers
375
views
tc traffic shaping with HTB and CQB causes packet transmission gap inconsistencies
I am sorry if this is duplicate of https://serverfault.com/q/1076769/822163.
I created that first and then realized the Linux and Unix stack exchange is the right place.
Problem: When the tc HTB or ...
7
votes
1
answer
2k
views
How to select based on an allowlist/blocklist using jq
Using jq, how can we select json elements from an array based on inclusion/exclusion of each element's key in some allowlist/blocklist?
I want to do a case-insensitive contains (so allowlist/blocklist ...
2
votes
1
answer
148
views
How to get a list of latest files recursively with a specific extension and copy them into a folder
How to get a list of the latest files with the extension .jmx recursively and use the file names in another process?
I have a list of JMX files organized into different folders and want to pick the ...
- 145
0
votes
1
answer
392
views
What does this bcf error mean?
I ran this code to filter out biallelic variants from my vcf file:
bcftools view --max-alleles 2 --exclude-types indels merged.ecDNA.vcf.gz>merged.ecDNA2.vcf.gz
However, I got an error message:
[E:...
5
votes
4
answers
4k
views
Extract values from a string that follow a specific word using sed
I'd like to extract only a specific value from command output.
The string that the command returns is something like this:
Result: " 5 Secs (11.2345%) 60 Secs (22.3456%) 300 Secs (33.4567%)"
...
- 63
-1
votes
1
answer
48
views
How to compile lines with a certain word in the last column into a separate file?
I have a csv file and I need to filter it out into two files based on whether the last column contains the word "ecDNA". I already have two more copies of the file to edit without changing ...
2
votes
3
answers
171
views
BASH Script - Reading output and cutting by value
I am trying to create a BASH script that will run a command, filter the output of that command and then read the results of that output to then print only sections that meet the given requirement.
For ...
user473736
0
votes
0
answers
35
views
Filter csv file with AWK passing a variable in the regex [duplicate]
I need to filter a very large csv file in the most efficient way. I already tried csvgrep but is not efficient timewise, so I'm now trying to use AWK for this task.
The csv file has a ; separator, and ...
0
votes
4
answers
65
views
reading and discarding lines
I have a question about a problem that I have to solve, I have my lines in this way:
Input
GTEX-1117F-0003-SM-58Q7G
GTEX-1117F-0003-SM-5DWSB
GTEX-111CU-0826-SM-5EGIJ
GTEX-111CU-0926-SM-5EGIK
GTEX-ZZPU-...
1
vote
1
answer
2k
views
Only show entries of the last hour of log file
I have a huge logfile access.log with entries like:
192.11.111.111 - - [05/Mar/2021:00:00:02 +0100] "GET ..."
192.250.14.80 - - [05/Mar/2021:00:00:09 +0100] "GET ..."
12.249.66.42 -...
- 11
0
votes
0
answers
2k
views
Linux tc filter add - errors RTNETLINK answers: Operation not supported
I'm working on the network simulation to create some traffic on specific ports , and trying to inject the network delay. I'm using the linux tc utility to do this operations. So i'm new to the tc ...
- 1
0
votes
1
answer
121
views
Filter directory tree and create bash menu from results
I have a Google Drive folder in the cloud that I sync with my server to backup and restore a WordPress web directory and MySQL database dump and I accomplish this with rclone. I am able to see the ...
- 57
0
votes
1
answer
195
views
BlueTooth device(output) and PulseAudio's module-echo-cancel help needed
I'm trying to run PulseAudio's 'module-echo cancel' filter on my bluetooth speaker(output only).
I've edited /etc/pulse/default.pa like that:
.ifexists module-echo-cancel.so
load-module module-echo-...
0
votes
2
answers
417
views
Limit SSH to a range of IP
I work with OpenBSD and I want to limit the port 22 for all user who have an IP between 192.168.42.40 - 192.168.42.60. But I don't know how to do this.
I need to modify the packet filter ?
(SSH is on ...
