Questions tagged [text-processing]
Manipulation or examining of text by programs, scripts, etc.
6,486
questions
0
votes
3answers
24 views
Extract one string between double quotations
I'm trying to make a custom polybar module that shows the current active window class, since the default module outputs the window name.
I'm using this command to get the window class:
xprop -id $(...
1
vote
3answers
71 views
Simple shell script unable to go through thousands of files; starts fine, but throws “unexpected EOF while looking for matching `”`" after some time
Shell Script in Question
Let me explain what I am trying to do by e.g. so you can understand better. Let's say I have 100 .torrent files in a directory. 2 of them will download xxx.epub and yyy.epub ...
0
votes
1answer
50 views
Bash script to comment and uncomment line
hope you are doing well during this time. The thing is I d like to comment some lines in my txt file. I have a code to comment and uncomment multiple lines, however I dont know how to get company name....
1
vote
2answers
49 views
Remove duplicates csv based on first value keeping the longest line between duplicates
I have many different csv files in a folder (megadrive.txt, snes.txt) like this:
Aerial Assault (USA);Aerial Assault (USA);Sega Master System;;;;;;;;;0;;;;;
Aerial Assault (USA);Aerial Assault (USA);...
0
votes
1answer
28 views
How to replace a matching string in one file with matching string in another file?
I have a file like this:
head cleandata.map
1 1:775852:T:C 0 775852
1 1:1120590:A:C 0 1120590
1 1:1145994:T:C 0 1145994
1 1:1148494:A:G 0 1148494
...
And another file:
head ...
2
votes
1answer
33 views
Replace specific character matches in all rows, using the first line as a reference
H, I want to replace specific character matches (.) in all rows, using the first line as a reference
My attempt at re-hashing an answer I got to a different question:
awk -F'|' 'BEGIN{OFS=FS} NR==1 ...
0
votes
0answers
48 views
How to output only certain lines from a file having the line numbers in another file? [duplicate]
I have a file with desired line numbers:
4988
5008
5028
5050
5072
5094
5116
5138
5160
5182
5204
5224
. etc .
These are line numbers indexes into a log file that I have:
URL : https://dc4-...
6
votes
7answers
1k views
How do I replace ALL text in a file after a certain line with the content of another text file, using sed/awk?
I am currently trying to automate updating the text of a file, titled original_file.txt. Imagine the file looks like the following:
common_text
### REPLACE EVERYTHING AFTER THIS LINE ###
...
1
vote
1answer
26 views
egrep, sed or awk to get the number of active dhcp leases?
Let's say I have the following dhcpd.leases file:
# The format of this file is documented in the dhcpd.leases(5) manual page.
# This lease file was written by isc-dhcp-4.3.6-P1
# authoring-byte-...
0
votes
0answers
13 views
Is there a way to pipe to a visual editor and then continue the pipeline? [duplicate]
I know that some command-line programs can open visual editors like Sublime in a "wait mode" which opens the editor, lets you edit, and on-close it resumes the program with the file contents (like git ...
-2
votes
4answers
62 views
Counting commutative columns in a file
I have pairwise strings like this in my file
A B
A C
B A
B C
C A
I'm looking for a way to count how many commutative pairs I got. I.e A B and B A makes one such pair but B C doesn't (because we miss ...
-2
votes
1answer
44 views
I want to extract information from my text file and save it in a separate file
I want to extract information from my file and save it in a separate file.
The input file looks like:
>Feature NODE_18_1000_cov_1.366138
1 888 CDS
db_xref COG:COG3385
inference ab initio ...
-1
votes
4answers
80 views
I want to replace texts in my file
My file looks like:
NODE_13_length_50291_cov_8.067223 GTTGCCTAATATCGCCAATCCAGCTCCT
NODE_25_length_22118_cov_8.258986 TATCGCCAATCCAGCTCCT
NODE_36_length_15273_cov_5.283743 TGCCTAATATCGCCAATCCAG
I ...
6
votes
6answers
314 views
Replace each character of white space at the end of each line with '_'
I'm wanting to replace each character of white space at the end of each line with '_'. I found a similar question and answer for leading whitespace. But have failed to reconstruct it for trailing ...
0
votes
4answers
108 views
find most common consecutive two word sequence awk
I have a text file named 'file'.
I need to find most common two word sequence using awk, while "this is" and "is this" are considered equal in our count.
sample text :
name my name is this is our ...
1
vote
2answers
69 views
How to add add '\n' to the end of each line in a log file
I'm using msmtp to create e-mail alerts on some RPi kiosks we have set up at my company, and unfortunately if I try to include the last 5 lines of a log file in the e-mail alert, msmtp doesn't handle ...
-1
votes
0answers
33 views
Merging two files in awk
The fields in both files are delimited by * and in both files, the first field is the same.
file 1:
D00106863615
..
D00626054332
(4k records)
file 2
D00106863615 *01*1*20161002*3**20100921*...
0
votes
1answer
35 views
Need program - “editor” that lets you check-mark lines to keep
I was wondering if anybody could recommend a program or plug-in (for an editor) to solve the following problem: I have a file with a large list (one line per item). I need to manually go through ...
2
votes
2answers
107 views
awk: replace a character on a given line and column if it doesn't match a character on the same given column in the first line
I've tried for ages and I haven't even gotten close. Using awk, how do I: replace every '*' with '-' on every line > 1 in every column, but only if the corresponding column in line 1 is not an '*'?
...
0
votes
1answer
58 views
Can one use backreference as replacement value in sed?
Source file:
Linux 2.6.32-754.18.2.el6.x86_64 (myhostname) 03/24/2020 _x86_64_ (64 CPU)
07:32:01 PM 57 25.79 0.00 4.99 0.00 0.00 0.00 0.02 0....
0
votes
1answer
25 views
I would like to analyze a TXT file, but the data is a 1 column array, and it needs to be a in a multi-column array
Is there a way for me to manipulate the data (i.e.; change some feature of, so that it looks like it should, a multi-column array) without having to manually go through it? There are thousands of rows ...
-1
votes
0answers
17 views
salt.state.file.replace: replacing entries in /etc/fstab
I would like to achieve the same as under sed: replacing entries in the /etc/fstab, but as salt.state.
How would you implement the following awk-command as salt.state.file.replace?
awk '$2 ~ "/tmp"{ ...
-6
votes
3answers
41 views
Modify text file with if-else statement
I have an input file like this:
3 1 10 1 6
9 4 2 4 1
9 2 2 2 1
2 4 2 2 2
9 4 9 2 9 1 2
The conditions that need to be followed:
If NF==5 and if $2 and $4 are the same then ...
1
vote
1answer
34 views
Group following fields by matched header line using Awk
I have an output of a cli tool that gives me a list like this one:
Listing accountBrand
[2020-03-24 18:03:42] 20200211204415-create-accountBrand.js : Initial creation of accountBrand
[2020-03-24 ...
0
votes
0answers
28 views
Read data from two CSV files dates different format and divide
I have two csv files one called data.csv and another called GBPUSD.csv
In the data.csv there are three columns Name , Date and Amount and in the GBPUSD.csv file there is Date & Price what I need ...
1
vote
5answers
58 views
Removing strings from beginning and end of .txt files in directory?
I have a set of 1-line .txt files in a directory. They all have the same basic format, which I have pasted below labelled 'input':
For every file in the directory, I would like to remove the string ...
0
votes
3answers
41 views
Removing text after a delimiter up to a certain point in Bash/Linux
target_id tpm tpm tpm tpm tpm tpm tpm tpm tpm tpm tpm
PB.1.1|1:6643-11779(-)|transcript/55100 9207.33 13266.4 15244.3 2844.27 2708.52 2731.91 3193.83 2675....
1
vote
1answer
71 views
Piping errors to my function causes command to be ignored after 1 failure
Context: I have a Bash Script that copies files
function log () {
read IN
if [ "$IN" == "" ]; then
:
else
echo "$datetime"$'\t'"$IN" | tee -a logfile
fi
}
function ...
0
votes
2answers
35 views
Is it possible to merge rows based on the composite unique values of two columns
Sorry I am very new to Linux and I am uncertain the capability of bash for what I want to achieve.
I want to merge column-2 values if column-1 and column-3 values are identical. In this case, if ...
0
votes
3answers
63 views
List the output in a table form
I have a output of
Student Name: abc
Roll Num: 123
Student Name: xyz
Roll Num: 124
and I need to print in below format
Student Name Roll Num
abc 123
xyz 124
can ...
0
votes
2answers
42 views
Merge multiple rows based on a single column value
I have an input file like this:
3 1 10 1 6
9 4 2 4 1
9 2 2 2 1
I want to merge two rows based on the 5th column value, the output would look like this:
3 1 10 1 6
9 4 2 4 9 2 2 ...
8
votes
2answers
2k views
What does $0=$1 mean in awk?
Let's say, I have an input text file (sample_simple.txt) like this:
3 1 10 1 6
9 4 2 4 1
9 2 2 2 1
By using the command: awk '$0=$1" "$1*$2" "$3*$4' sample_simple.txt, I get the following output:
...
-2
votes
0answers
33 views
Remove a field when numeric
Delete field 3 and field 4 only when field 3 is numeric
before
172.16.230.160 wsk 172.15.230.160 wsk read 0 written 4442728
after
172.16.230.160 wsk read 0 written 4442728
0
votes
3answers
44 views
Linux pattern searching
If I have a file with lots of directory info like:
/home/svdev/src/lib
/home/dev/project/doc
/home/server/etc
I want to get the directory name next to "/home" directory. So for this set of data, my ...
0
votes
1answer
48 views
How to add the content of file A into file B between the pattern match
I have two files:
This is the content of file A:
etc...
this is a test file having \@ and \# and \$ as well
looking for awk or sed solution to print this content in another file between the matching ...
0
votes
2answers
79 views
How to regroup string with same length using awk
I have a long list of strings like this:
_ah_
_asn_
_ai_
_errr_
_an_
And I would like to transform it like to this
"_ah_ai_an_",
"_asn_",
"_errr_"
I have tried :
cat file | awk '{ print length, $...
0
votes
1answer
21 views
Split file based on beginning and end XML tag
Can split be used to split a file into multiple segments where the size of each segment is based on the number of matches of an XML element ?
For example for below XML split when encounters of "<...
0
votes
1answer
43 views
Find consecutive lines with specific strings and modify file based on table
I have a text manipulation problem that I have been unable to solve. Say I have a text file like the one below (text.txt). There will be cases in which a line with /locus_tag is followed by a line ...
-1
votes
3answers
39 views
Insert lines after character
I have data in a long lined file example:
<currency>USD</currency><amount>111</amount><amountf>111/amountf><recv_confirms>2</recv_confirms><fee>22&...
-1
votes
2answers
57 views
awk interpolation of values in column
i'm facing a problem with looong txt files i have this exemple :
542 701.00
789
971 703.00
123 708.00
879
785
974
101 716.00
and i need to replace the missed values in column ...
0
votes
3answers
35 views
How can I display the first n different lines in two files from a shell?
How can I display the first n different lines in two files from a shell ? I've tried with grep -vf but it does not really work.
Suppose n = 5
This input:
file1
a
b
c
d
e
f
g
h
i
j
k
l
m
n
o
file2
...
0
votes
3answers
78 views
Regular relational operator '<' not working as expected awk
I have a file like this:
0.2
0.2
0.2
0.2
0.2
0.2
0.2024
0.2025
0.2027
0.2027
0.2029
0.2059
0.2059
0.2059
0.2059
0.2099
0.2099
0.2099
0.2105
0.2113
0.2113
0.2195
0.2198
0.2206
0.2206
0.2206
0.2989
0....
-1
votes
3answers
54 views
How can I check if same text appears in two different files
I want to check (trick or any shortcut) whether text-string exists in two files.
The content of file one is
a.txt
b.txt
c.txt
d.txt
The content of file two is
c.txt
a.txt
d.txt
How I can check if ...
0
votes
0answers
55 views
How to ensure that list order stays the same, when lines are missing?
I run into a problem and can't wrap my head around it. Any help is welcome.
Example
Correct order, given or in external file:
red blue green brown black
Input:
red
blue
green
black
Output:
red ...
-1
votes
2answers
51 views
Capture only the file name from the directory Path
Requirement
Col1 , Col2 , Col3 , Col4
1 , a:/b/c/abc.txt , f:/b/c/abc1.txt , a:/b/c/abc6.txt
2 , b:/bd/a/s/3/d/mon.dat , g:/b/c/abc 2.txt, b:/b/c/ab7....
1
vote
1answer
42 views
Remove newlines using shell command (must work in a KDE global shortcut)
I need a shortcut to strip newlines from the text in the current selection. I want to put my command into a KDE global shortcut, which I already know how to do. The command I came up with is:
sh -c '...
0
votes
1answer
16 views
XML read column and add new data based off date format
I have the following data in a XML document
NAME 19/02/2019 23/02/2019 23/02/2019 13:21 12.99 0 12.99 Paid PayPal
NAME 19/02/2019 23/02/2019 23/02/2019 13:22 12.99 0 12....
0
votes
3answers
76 views
one shell command to find every n-gram works in text
I have a text stream or a file that contains words separated by whitespace.
Like:
I have a toy. you may not like it.
Every whitespace-separated word may be composed by two or more small words, ...
0
votes
1answer
67 views
Copy the records between header and trailer into a new file including header and trailer [closed]
I have a file which has multiple headers,detail and trailer records.(fixed length)
For instance my file looks like
Habcd20200317
R123 abc def
R234 dfg thg
T2 2018907
Hagfd20200316
R346 adf thg
R456 ...
0
votes
7answers
93 views
Struggling with GREP commands, removing decimal point correctly
I've written the shell command below to output the CPU's clock speed as an integer:
grep -m 1 'cpu MHz' /proc/cpuinfo | grep -o -E '[0-9]+'
OUTPUT:
900
063
This is because the exact clock speed is ...
