Questions tagged [text-processing]
Manipulation or examining of text by programs, scripts, etc.
7,210
questions
-2
votes
1answer
17 views
merging two files according to a common column; keep the value if the variant is present in file 2, and add 0 to those which are not present in file 2
I would like to merge two files, which share some common data. File 1 includes more than file 2. I want to merge the files based on their shared column (3) following the order of file 1 and I want to ...
0
votes
5answers
52 views
How to print in only one column if condition with awk
I have a file bla.tsv (FS=\t):
id hub fil plop drift
3 187 0.91 bis XX
2 245 0.93 XX
4 250 0.70 bis XX
1 245 0.95 bis XX
And I want to print in the column plop the folowing string :
If hub<...
0
votes
4answers
67 views
Grep for a range of numbers
I have a .txt file with multiple lines that gives amino acid and residue data. The data looks like this:
ARG262-Side ASP368-Side 140,83%
ARG95-Side GLU107-Side 103,73%
ARG474-Side VAL468-Main 94,93%
...
0
votes
2answers
30 views
Parse Apache logs from command line (logs with multiple delimiters)
How to parse /var/log/apache2/other_vhosts_access.log and get the value of each column from command line?
This is surely a very common problem, and I tried with awk, but it's not as simple as it looks,...
0
votes
2answers
36 views
search and replace with sed special character
I need to perform search and replace using sed.
starting from this sample text for example :
<TextView
android:textSize="20.0sp"
android:...
0
votes
1answer
30 views
Use grep to find any lines that have the letter a, e or i followed by the letter o or u [closed]
grep [aei].*[ou]
This command is wrong.
-1
votes
0answers
25 views
filter ignore lines in a big text file
i generated a big text file input.txt, the content is lines of file paths, which i will use later in a loop
but there're some path patterns i want to remove, like node_modules
i have the ignore ...
5
votes
3answers
250 views
Adding a string to every column in a line- excluding the first column, variable line length, the string is contained in the first column of each line
I have a file with multiple lines/rows, and each line contains a variable amount of columns:
Name1 String111 String112
Name2 String121 String122 String123
Name3 String131 String132 String133 String134
...
0
votes
1answer
50 views
how to replace spaces in a string with '\ ' in bash
I'm writing a bash script that prints the contents of a file called website (using cat) that is inside a folder with a space in it's name, for example folder 1. but I can't make bash get the path ...
1
vote
3answers
55 views
sed insert: preserve leading spaces from inserted string
I have a variable (extracted from a diff file result) with leading spaces that I want to insert into a file at a specific line. The variable:
add=" public function add($hardware_id = null)&...
1
vote
6answers
125 views
modify lines when find specified string
I have a file which looks like this
ABC
2 3 4
7 9 4
1 2 5
ABC
13 11 17
2 1 1
ABC
7 9 14
5 8 2
9 9 9
7 1 2
and I want to ...
-1
votes
1answer
46 views
How to merge one column from two different file using awk
I have two file bla.tsv (FS=\t) and blo.tsv (FS=\t) generate by the same code.
bla.tsv:
id hub klos
3 2 bis
2 5
4 2 bis
5 2
8 3
7 2 bis
blo.tsv:
id hub klos
3 2
2 5 jus
4 2
5 2
8 3 jus
7 2
...
0
votes
1answer
47 views
Merging two files based on a common column; add 0 for variants not present in file 2, and keep the original value in those which are present
I would like to merge two files, which share some common data. File 1 includes more than file 2. I want to merge the files based on their shared column (3) following the order of file 1 and I want to ...
2
votes
7answers
362 views
Transform the text representation of a timespan in shell script
I have a program that outputs a time in the format
2s576039936ns
I need to convert this to the following format:
2.576039936
How can I do this in shell script?
-4
votes
3answers
74 views
Remove duplicate lines with possibly different spacing, but ignore lines that start with #
This is a follow-up to an earlier question. I use this syntax in order to remove duplicate lines even if the fields are spaced differently
awk '{$1=$1};!NF||!seen[$0]++' /tmp/fstab
Now I want to ...
3
votes
3answers
402 views
How to remove duplicate lines ignoring tab or spaces
I want to remove duplicate lines from /etc/fstab, so I did this:
awk '!NF || !seen[$0]++' /etc/fstab > /etc/fstab.update
UUID=3de0d101-fba7-4d89-b038-58fe07295d96 /grid/sdb ext4 defaults,...
0
votes
1answer
40 views
Attempting to turn system config into an easier to compare format with awk or sed [closed]
Does anyone know what the term for this style of config is, where sub-sections are declared by tabs with an exit or up indicating the end of the section:
config
system
security
...
0
votes
2answers
35 views
Add “block prefix” to all lines belonging to the same block of a file
I tried to make the title as specific as possible, an example should clear the problem.
I have a tab-separated file with lines starting with 1 to 5. That is considered as a block (each block can have ...
-1
votes
0answers
21 views
find files that start with the same number in a folder
I have a folder that contains files that starts with a number, as follows:
1__File.sql
2__File_differentName.sql
3__Maybe_Something_else.sql
3__This_is_Not_Allowed.sql
I want to find if two files ...
0
votes
0answers
18 views
Count lines based on number of CSV fields [closed]
How can I count the lines that contain only one comma separated and ignore count the lines with two comma separated in text file?
Below is my code
def countLines(filename):
with open(filename,'r') ...
1
vote
1answer
38 views
merging two files according to a common column and add 0 to those which are not present in file 2
I would like to merge two files, which share some common data. File 1 includes more than file 2. I want to merge the files based on their shared column following the order of file 1 and I want to add ...
0
votes
1answer
42 views
remove every instance of new line between two patterns
I have a text file with multiple instances of strings like this:
Query= XYZL01000001.1_wekjasdlpoa_strain_NLLLRL_WC-
3ssss66_P2sss0contig1.1__whole_genome_shotgun_sequence
Length=239131
which I need ...
0
votes
1answer
41 views
Using grep to find two or more vowels in a row [duplicate]
So far I am using grep -E '[aeiou]{2}' /usr/share/dict/american-english. This is giving me two vowels, but I want them to be only vowels in a row, like "tooth", "beer", etc.
So I ...
-3
votes
5answers
78 views
Delete everything before a pattern, and removing symbol
I have over 300 lines containing text from Jenkins plugins in this pattern:
Server Sent Events (SSE) Gateway Plugin (sse-gateway): 1.24
Common API for Blue Ocean (blueocean-commons): 1.24.4
Handy Uri ...
-1
votes
0answers
45 views
Remove extra commas from a CSV file specific occurrence [closed]
We get in dozens of .csv files that should all have 16 commas on each line. But occasionally we will receive a csv file where there are 17 commas, which throws off our import routine.
I need to be ...
0
votes
7answers
92 views
How to remove double quotes when it comes within double quotes
I have a data like this
"Apple grape","Banana"orange","Red green"
"Apple grape","Bananaorange","Red green"
I want to remove the single ...
0
votes
3answers
44 views
Combine two files using common column
I have 2 files.
File 1 is:
burnj01,Joe Burns
steves02,Santosh Steve
File 2 is:
burnj01,0001
burnj01,0002
burnj01,0010
burnj,0100
burnj01,2000
steves02,2048
steves02,2049
steves02,2091
and I would ...
-1
votes
1answer
49 views
sed command to replace multiple lines after interval [closed]
I would like to replace the first character C with H for 23 lines (say, lines 83 to 106) after every interval of 82 lines. How can I do this using sed or awk?
1
vote
1answer
87 views
How to compare two column from two files with specific condition in awk
I have a data file A.tsv (field separator = \t) :
id clade mutation
243 40A titi,xixi,lolo
254 20B titi,toto,jiji,lala
261
267 20B lala,jiji,jojo
and a template file B.tsv (field separator = \t) :
...
0
votes
1answer
33 views
Copy line matching pattern in multiple files to a new file
I have 200 output files, of which I need to compare one line.
The exact line varies, but Total Energy = xyz always exists.
Is there an easy way to copy that line from my output files in a new file? I'...
0
votes
1answer
42 views
Why Windows text files encoding crash after I open them in Linux system?
I use Debian and sometimes I need to mount Windows disks because I want to read some *.txt file from Windows. Sometimes it's okay, but sometimes after I edit or read file using Debian system and then ...
-6
votes
0answers
32 views
Extract all transcript which contains <20 exons in a transcript [closed]
The first 3 column contains the genome coordinates 4th column contains transcript name 5th column contains exon name and 6th column contain exon rank. So I want to separate this file into 2 files: 1st ...
0
votes
2answers
94 views
Replace spaces in certain positions
I have a text file as below.
data.txt
12345678909876456 qwerty
1234567890987 qwerty
23456789011234567 qwerty
I want to show it in the below way.
It will replace from 11th position to 20th ...
5
votes
5answers
712 views
Replace value in “key: value” statement, but only on first occurence of the key in the file
I have a yml file
spring:
datasource:
url: url
username:test
password: testpwd
api:
security:
username:foo
password: foopwd
I want to update only the first occurrence or ...
0
votes
1answer
43 views
Linux sed command replace regex string
I'm trying to edit a text file programmatically, which contains a line like this:
db.username="tata"
db.password="toto"
which I want to look like this:
db.username="fofo"...
-1
votes
2answers
64 views
Help me to get command or ansibile Regex to fetch below details
My data:
KVM05786Y: The Communication host "ADMK05786HTREDIRECT_JA3A" is SUCESS
KVM05786Y: The Communication host "KOPYUTRCEPORT_JA3A" is SUCESS
KVM05786Y: The Communication host &...
2
votes
3answers
72 views
How can I use sed or tr to delete strings like “[digits]”?
I recently had to handle lots of downloaded paper, which included many annoying "[...]" in the text. Like this:
Local delivery of a wide selection of growth factors [e.g. platelet-derived ...
-4
votes
6answers
94 views
Duplicate lines containing a string with edits [closed]
I need to find a way to duplicate lines in a file containing a string with small edits. The dupe would have to show up above the original line. Would sed or awk be my best bet?
Ideally replacing ...
0
votes
2answers
51 views
How to change integer in bash script according to written .dat filenames
In my bash script, I want to update a string according to new written filenames with .dat format. Here is what I am trying to do:
For example, I use a file named blabla_3200.dat in my bash script, ...
1
vote
1answer
50 views
How to create folders based on part of a filename and also extract & rename specific columns using awk
the following code extracts data from csv files contained in a folder, creates new folders (based on filename and date), creates and stores multiple csv files in the said folders. more info on this is ...
0
votes
1answer
28 views
Want to sort file according to UTC date yyyy-mm-ddThh:mm:ss.000Z
I want to sort dates according to 4th column I tried sort -k4. It's not working.
Example of the input file:
| 2021-02-17T15:38:53.000Z | | 83027fed-1570-4b39-ab4e-3f75f126cf1c | 2021-02-17T16:02:32....
2
votes
1answer
54 views
Does busybox have a tool for dumping 4 byte words as hex on a little endian machine? [closed]
I am trying to use busybox to dump files in /sys and /proc.
I happen to have an embedded device with busybox 1.31.
I noticed that xxd does not have the -e option for little endian.
I can't figure out ...
0
votes
1answer
17 views
In what files/form are the “Search actions” of Whisker Menu saved?
Whisker Menu has a great but quite underrated feature called "Search Actions" that can easily trigger a predefined command to search/open/run various folders/files/programs very quickly by ...
2
votes
2answers
73 views
How to compare two column of two file and print not matching pattern with awk
I have a data file A.tsv (field separator = \t) :
id clade mutation
243 40A siti,toto,mumu
254
267 40B lala,sisi,sojo
and a template file B.tsv (field separator = \t) :
40A toto,xixi,saxa
40B ...
0
votes
1answer
49 views
echo $’hi\nhi’ > /etc/list.list not working
When I use echo $’hi\nhi’ > /etc/list.list in an executable script text file I create comprising multiple commands on Kali Linux 2020.4 Live USB it:
Includes the $ in the files text while ...
-1
votes
5answers
140 views
Sed delete empty lines after a particular line number [duplicate]
I am trying to delete all empty lines after line number 3, till the end of the file:
cat ${File}
1
2
3
4
5
5
6
7
8
9
sed -e '3,${s~^$~~g}' ${File}
1
2
3
4
5
5
6
7
8
9
Observation: No ...
1
vote
4answers
104 views
How to compare two column of two file and print the number of match with awk
I have a data file A.tsv (field separator = \t) :
id clade mutation
243 40A titi,toto,lala
254
267 40B lala,jiji,jojo
and a template file B.tsv (field separator = \t) :
40A lala,toto,xixi,...
4
votes
2answers
154 views
How to split CSV file and create multiple CSV files based on a column
I have a csv file with the format:
Input.csv:
TIMESTAMP,Data1,Data2,Data3,Data4
"2021-01-03 00:00:00",80953,3.243183,2.943338,358.0123
"2021-01-03 00:01:00",80954,2.173187,1.990327,...
-1
votes
2answers
53 views
How to merge two files based on the matching of one column?
I have two files, B.csv:
1,AD
2,AB
3,AC
5,AF
7,AE
and C.csv:
1,x
3,z
5,y
How do I get this output:
1,AD,x
2,AB,
3,AC,z
5,AF,y
7,AE,
by matching the common column 1 in both of the files?
0
votes
2answers
63 views
How to combine data from two file with the header in awk
I have two files, A.tsv and B.tsv:
A.tsv (field separator = \t):
Sample ID Internal Control Result Consensus
4686427 Pass Not Detected Not Available
4666275 Pass Detected Not ...
