Questions tagged [text-processing]

Manipulation or examining of text by programs, scripts, etc.

Filter by
Sorted by
Tagged with
-2
votes
1answer
17 views

merging two files according to a common column; keep the value if the variant is present in file 2, and add 0 to those which are not present in file 2

I would like to merge two files, which share some common data. File 1 includes more than file 2. I want to merge the files based on their shared column (3) following the order of file 1 and I want to ...
0
votes
5answers
52 views

How to print in only one column if condition with awk

I have a file bla.tsv (FS=\t): id hub fil plop drift 3 187 0.91 bis XX 2 245 0.93 XX 4 250 0.70 bis XX 1 245 0.95 bis XX And I want to print in the column plop the folowing string : If hub<...
0
votes
4answers
67 views

Grep for a range of numbers

I have a .txt file with multiple lines that gives amino acid and residue data. The data looks like this: ARG262-Side ASP368-Side 140,83% ARG95-Side GLU107-Side 103,73% ARG474-Side VAL468-Main 94,93% ...
0
votes
2answers
30 views

Parse Apache logs from command line (logs with multiple delimiters)

How to parse /var/log/apache2/other_vhosts_access.log and get the value of each column from command line? This is surely a very common problem, and I tried with awk, but it's not as simple as it looks,...
0
votes
2answers
36 views

search and replace with sed special character

I need to perform search and replace using sed. starting from this sample text for example : <TextView android:textSize="20.0sp" android:...
0
votes
1answer
30 views
-1
votes
0answers
25 views

filter ignore lines in a big text file

i generated a big text file input.txt, the content is lines of file paths, which i will use later in a loop but there're some path patterns i want to remove, like node_modules i have the ignore ...
5
votes
3answers
250 views

Adding a string to every column in a line- excluding the first column, variable line length, the string is contained in the first column of each line

I have a file with multiple lines/rows, and each line contains a variable amount of columns: Name1 String111 String112 Name2 String121 String122 String123 Name3 String131 String132 String133 String134 ...
0
votes
1answer
50 views

how to replace spaces in a string with '\ ' in bash

I'm writing a bash script that prints the contents of a file called website (using cat) that is inside a folder with a space in it's name, for example folder 1. but I can't make bash get the path ...
1
vote
3answers
55 views

sed insert: preserve leading spaces from inserted string

I have a variable (extracted from a diff file result) with leading spaces that I want to insert into a file at a specific line. The variable: add=" public function add($hardware_id = null)&...
1
vote
6answers
125 views

modify lines when find specified string

I have a file which looks like this ABC 2 3 4 7 9 4 1 2 5 ABC 13 11 17 2 1 1 ABC 7 9 14 5 8 2 9 9 9 7 1 2 and I want to ...
-1
votes
1answer
46 views

How to merge one column from two different file using awk

I have two file bla.tsv (FS=\t) and blo.tsv (FS=\t) generate by the same code. bla.tsv: id hub klos 3 2 bis 2 5 4 2 bis 5 2 8 3 7 2 bis blo.tsv: id hub klos 3 2 2 5 jus 4 2 5 2 8 3 jus 7 2 ...
0
votes
1answer
47 views

Merging two files based on a common column; add 0 for variants not present in file 2, and keep the original value in those which are present

I would like to merge two files, which share some common data. File 1 includes more than file 2. I want to merge the files based on their shared column (3) following the order of file 1 and I want to ...
2
votes
7answers
362 views

Transform the text representation of a timespan in shell script

I have a program that outputs a time in the format 2s576039936ns I need to convert this to the following format: 2.576039936 How can I do this in shell script?
-4
votes
3answers
74 views

Remove duplicate lines with possibly different spacing, but ignore lines that start with #

This is a follow-up to an earlier question. I use this syntax in order to remove duplicate lines even if the fields are spaced differently awk '{$1=$1};!NF||!seen[$0]++' /tmp/fstab Now I want to ...
3
votes
3answers
402 views

How to remove duplicate lines ignoring tab or spaces

I want to remove duplicate lines from /etc/fstab, so I did this: awk '!NF || !seen[$0]++' /etc/fstab > /etc/fstab.update UUID=3de0d101-fba7-4d89-b038-58fe07295d96 /grid/sdb ext4 defaults,...
0
votes
1answer
40 views

Attempting to turn system config into an easier to compare format with awk or sed [closed]

Does anyone know what the term for this style of config is, where sub-sections are declared by tabs with an exit or up indicating the end of the section: config system security ...
0
votes
2answers
35 views

Add “block prefix” to all lines belonging to the same block of a file

I tried to make the title as specific as possible, an example should clear the problem. I have a tab-separated file with lines starting with 1 to 5. That is considered as a block (each block can have ...
-1
votes
0answers
21 views

find files that start with the same number in a folder

I have a folder that contains files that starts with a number, as follows: 1__File.sql 2__File_differentName.sql 3__Maybe_Something_else.sql 3__This_is_Not_Allowed.sql I want to find if two files ...
0
votes
0answers
18 views

Count lines based on number of CSV fields [closed]

How can I count the lines that contain only one comma separated and ignore count the lines with two comma separated in text file? Below is my code def countLines(filename): with open(filename,'r') ...
1
vote
1answer
38 views

merging two files according to a common column and add 0 to those which are not present in file 2

I would like to merge two files, which share some common data. File 1 includes more than file 2. I want to merge the files based on their shared column following the order of file 1 and I want to add ...
0
votes
1answer
42 views

remove every instance of new line between two patterns

I have a text file with multiple instances of strings like this: Query= XYZL01000001.1_wekjasdlpoa_strain_NLLLRL_WC- 3ssss66_P2sss0contig1.1__whole_genome_shotgun_sequence Length=239131 which I need ...
0
votes
1answer
41 views

Using grep to find two or more vowels in a row [duplicate]

So far I am using grep -E '[aeiou]{2}' /usr/share/dict/american-english. This is giving me two vowels, but I want them to be only vowels in a row, like "tooth", "beer", etc. So I ...
-3
votes
5answers
78 views

Delete everything before a pattern, and removing symbol

I have over 300 lines containing text from Jenkins plugins in this pattern: Server Sent Events (SSE) Gateway Plugin (sse-gateway): 1.24 Common API for Blue Ocean (blueocean-commons): 1.24.4 Handy Uri ...
-1
votes
0answers
45 views

Remove extra commas from a CSV file specific occurrence [closed]

We get in dozens of .csv files that should all have 16 commas on each line. But occasionally we will receive a csv file where there are 17 commas, which throws off our import routine. I need to be ...
0
votes
7answers
92 views

How to remove double quotes when it comes within double quotes

I have a data like this "Apple grape","Banana"orange","Red green" "Apple grape","Bananaorange","Red green" I want to remove the single ...
0
votes
3answers
44 views

Combine two files using common column

I have 2 files. File 1 is: burnj01,Joe Burns steves02,Santosh Steve File 2 is: burnj01,0001 burnj01,0002 burnj01,0010 burnj,0100 burnj01,2000 steves02,2048 steves02,2049 steves02,2091 and I would ...
-1
votes
1answer
49 views

sed command to replace multiple lines after interval [closed]

I would like to replace the first character C with H for 23 lines (say, lines 83 to 106) after every interval of 82 lines. How can I do this using sed or awk?
1
vote
1answer
87 views

How to compare two column from two files with specific condition in awk

I have a data file A.tsv (field separator = \t) : id clade mutation 243 40A titi,xixi,lolo 254 20B titi,toto,jiji,lala 261 267 20B lala,jiji,jojo and a template file B.tsv (field separator = \t) : ...
0
votes
1answer
33 views

Copy line matching pattern in multiple files to a new file

I have 200 output files, of which I need to compare one line. The exact line varies, but Total Energy = xyz always exists. Is there an easy way to copy that line from my output files in a new file? I'...
0
votes
1answer
42 views

Why Windows text files encoding crash after I open them in Linux system?

I use Debian and sometimes I need to mount Windows disks because I want to read some *.txt file from Windows. Sometimes it's okay, but sometimes after I edit or read file using Debian system and then ...
-6
votes
0answers
32 views

Extract all transcript which contains <20 exons in a transcript [closed]

The first 3 column contains the genome coordinates 4th column contains transcript name 5th column contains exon name and 6th column contain exon rank. So I want to separate this file into 2 files: 1st ...
0
votes
2answers
94 views

Replace spaces in certain positions

I have a text file as below. data.txt 12345678909876456 qwerty 1234567890987 qwerty 23456789011234567 qwerty I want to show it in the below way. It will replace from 11th position to 20th ...
5
votes
5answers
712 views

Replace value in “key: value” statement, but only on first occurence of the key in the file

I have a yml file spring: datasource: url: url username:test password: testpwd api: security: username:foo password: foopwd I want to update only the first occurrence or ...
0
votes
1answer
43 views

Linux sed command replace regex string

I'm trying to edit a text file programmatically, which contains a line like this: db.username="tata" db.password="toto" which I want to look like this: db.username="fofo"...
-1
votes
2answers
64 views

Help me to get command or ansibile Regex to fetch below details

My data: KVM05786Y: The Communication host "ADMK05786HTREDIRECT_JA3A" is SUCESS KVM05786Y: The Communication host "KOPYUTRCEPORT_JA3A" is SUCESS KVM05786Y: The Communication host &...
2
votes
3answers
72 views

How can I use sed or tr to delete strings like “[digits]”?

I recently had to handle lots of downloaded paper, which included many annoying "[...]" in the text. Like this: Local delivery of a wide selection of growth factors [e.g. platelet-derived ...
-4
votes
6answers
94 views

Duplicate lines containing a string with edits [closed]

I need to find a way to duplicate lines in a file containing a string with small edits. The dupe would have to show up above the original line. Would sed or awk be my best bet? Ideally replacing ...
0
votes
2answers
51 views

How to change integer in bash script according to written .dat filenames

In my bash script, I want to update a string according to new written filenames with .dat format. Here is what I am trying to do: For example, I use a file named blabla_3200.dat in my bash script, ...
1
vote
1answer
50 views

How to create folders based on part of a filename and also extract & rename specific columns using awk

the following code extracts data from csv files contained in a folder, creates new folders (based on filename and date), creates and stores multiple csv files in the said folders. more info on this is ...
0
votes
1answer
28 views

Want to sort file according to UTC date yyyy-mm-ddThh:mm:ss.000Z

I want to sort dates according to 4th column I tried sort -k4. It's not working. Example of the input file: | 2021-02-17T15:38:53.000Z | | 83027fed-1570-4b39-ab4e-3f75f126cf1c | 2021-02-17T16:02:32....
2
votes
1answer
54 views

Does busybox have a tool for dumping 4 byte words as hex on a little endian machine? [closed]

I am trying to use busybox to dump files in /sys and /proc. I happen to have an embedded device with busybox 1.31. I noticed that xxd does not have the -e option for little endian. I can't figure out ...
0
votes
1answer
17 views

In what files/form are the “Search actions” of Whisker Menu saved?

Whisker Menu has a great but quite underrated feature called "Search Actions" that can easily trigger a predefined command to search/open/run various folders/files/programs very quickly by ...
2
votes
2answers
73 views

How to compare two column of two file and print not matching pattern with awk

I have a data file A.tsv (field separator = \t) : id clade mutation 243 40A siti,toto,mumu 254 267 40B lala,sisi,sojo and a template file B.tsv (field separator = \t) : 40A toto,xixi,saxa 40B ...
0
votes
1answer
49 views

echo $’hi\nhi’ > /etc/list.list not working

When I use echo $’hi\nhi’ > /etc/list.list in an executable script text file I create comprising multiple commands on Kali Linux 2020.4 Live USB it: Includes the $ in the files text while ...
-1
votes
5answers
140 views

Sed delete empty lines after a particular line number [duplicate]

I am trying to delete all empty lines after line number 3, till the end of the file: cat ${File} 1 2 3 4 5 5 6 7 8 9 sed -e '3,${s~^$~~g}' ${File} 1 2 3 4 5 5 6 7 8 9 Observation: No ...
1
vote
4answers
104 views

How to compare two column of two file and print the number of match with awk

I have a data file A.tsv (field separator = \t) : id clade mutation 243 40A titi,toto,lala 254 267 40B lala,jiji,jojo and a template file B.tsv (field separator = \t) : 40A lala,toto,xixi,...
4
votes
2answers
154 views

How to split CSV file and create multiple CSV files based on a column

I have a csv file with the format: Input.csv: TIMESTAMP,Data1,Data2,Data3,Data4 "2021-01-03 00:00:00",80953,3.243183,2.943338,358.0123 "2021-01-03 00:01:00",80954,2.173187,1.990327,...
-1
votes
2answers
53 views

How to merge two files based on the matching of one column?

I have two files, B.csv: 1,AD 2,AB 3,AC 5,AF 7,AE and C.csv: 1,x 3,z 5,y How do I get this output: 1,AD,x 2,AB, 3,AC,z 5,AF,y 7,AE, by matching the common column 1 in both of the files?
0
votes
2answers
63 views

How to combine data from two file with the header in awk

I have two files, A.tsv and B.tsv: A.tsv (field separator = \t): Sample ID Internal Control Result Consensus 4686427 Pass Not Detected Not Available 4666275 Pass Detected Not ...

1
2 3 4 5
145