Questions tagged [text-processing]
Manipulation or examining of text by programs, scripts, etc.
5,967
questions
1
vote
4answers
41 views
How to use sed get version number
How do I get the version number using sed
r-blob-1.0.2-abcsomething
urllib3-2.1.4-xyzsomething
cfile-8.0-dddsomething
How can I use "sed" to cut where there is - and [0-9], but start cutting after ...
0
votes
0answers
32 views
Merge two files searching by key and add at the end
I would merge data from two files into one adding only interesting Values from column. Searching key is UID in column 3 File #1. Also it should remove/omit UID duplicates and empty lines if appear.
...
0
votes
0answers
38 views
How can I remove all cases of a strings on terminal? [on hold]
So I have a dataset with mostly numerical values, but there are some strings categorizing the lines.
Example Data:
yeet 0 0 0 0 0 0 0
gg2 0 0 ...
5
votes
2answers
64 views
Column deletion based on number of string matches within column
I need a command that will remove any columns in a text file if they have =>${MaxAllowedNumberOfFs} 'F's within the column (a column that will have a varying number of rows).
I have some pseudo code ...
1
vote
0answers
28 views
Histogram with FASTA file
I am new to Linux.
I have a FASTA file that looks something like this:
>scaffold1
AAGACATAATATTTTGGAGGAATTAAAAAATTTAAGATGTATTTTATTATACATGTATTTTATTTATAACATAAATAAACATCCCAAGGAAAAGCAGTAGCT
>...
1
vote
1answer
19 views
How to search and replace for multiple words in one shot
I need to search and replace multiple words. For example, in my file, I want to search for any name of week days that appears with a comma after it: Sat, or Sun, or Mon, or Tue, or Wed, or Thur, or ...
0
votes
5answers
54 views
Remove n character after specific word
MY Data:
7468656.xml-87-
7468656.xml-88-
7468656.xml-89-
7468656.xml-90-
7468656.xml-91-
7468656.xml-92-
7468656.xml-93-
7468656.xml:94:
7468656.xml-115-
7468656.xml-116-
7468656.xml-117-
7468656.xml-...
2
votes
4answers
92 views
More elegant approach to append text to a file only if the String doesn't exist?
I want to append NoDisplay=true to a .desktop file, but only if the entry doesn't exist. I do this the following way:
grep -q 'NoDisplay=true' '/usr/share/applications/yelp.desktop' || bash -c 'echo "...
0
votes
1answer
68 views
Sed to replace string of 10 characters with 'XXXX ' [duplicate]
I have file which looks like
ACCINTCCADVANCESC
ACCINTCCADVANCESE
ACCINTCCADVANCESW
I want to replace INTCCADVAN with XXXX␣␣␣␣␣␣, the ␣s are spaces.
Specifically I want to replace a column with more ...
0
votes
5answers
40 views
add specific string at the beginning and at the end of line 1
The input file looks like this
H1 O1 C1 H2 H3 C2 H4 H5 O2 C3 H6 H7 C4 H8 H9 O3 C5 H10 H11 C6 H12 ......
I would like to have the following output file
z(A) H1 O1 C1 H2 H3 C2 H4 H5 O2 C3 H6 H7 C4 H8 ...
0
votes
2answers
30 views
multiply specific column in a file which consists of thousands columns
The in file looks like this
-17.3644 0.00000000 0.00000000 ....
-17.2703 0.00000000 0.00000000 ....
-17.1761 0.00000000 0.00000000 ....
-16.5173 0.00000000 0....
1
vote
3answers
54 views
How to extract domain name from email address and add it to a new column in a pipe delimited file?
I have a pipe-delimited file with an email address in the third column. I need help to create the fourth column out of the domain name from the email address in the third column using awk or sed ...
2
votes
2answers
64 views
Command for converting rows to CSV file
I have a file of the format, with a leading space before each line:
"Western Overseas",
"Western Overseas",
"^",
"--",
"^",
"--",
"--",
null,
24995,
9977,
"CR",
"Western Refrigeration ...
0
votes
2answers
29 views
Split delimited string using variable substitution
I have a sample file the lines of which are delimited by "#" and it contains 3 fields.
A#B#C
D#E#F
Can I use UNIX variable substitution instead of the following to get values of each field in the ...
2
votes
2answers
38 views
Split into multiple files by using unique columns in a unix file
I have requirement to split below file (sample.csv) into multiple files by using the unique columns (first 3 are unique columns)
sample.csv
123|22|56789|ABCDEF|12AB34|2019-07-10|2019-07-10|443.3400|...
-1
votes
4answers
50 views
Copying first line of text to another text file
Before:
file1
123
456
file2
foo
testing
After:
file1
123
456
file2
foo
testing 123
Any help would be greatly appreciated.
1
vote
3answers
75 views
Partition a file based on one column
I have the following data set:
M1 1 1233
M2 1 3212
M3 1 55323
M4 1 4444233
M5 1 23444
M6 1 555333
M7 1 55567
M8 2 22224
M9 2 55566
M10 2 4567
M11 3 44242
M12 3 234234
...
0
votes
1answer
49 views
Filtering a .txt file using a regular expression and a second file
I have a file.txt (it does not have the same number of columns for each row):
e.g.
1 21:10 21:23
2 1:94 1:100 1:123
3 14:1 14:60 14:23
I have another file (file2.txt) that contains 4 columns (...
0
votes
2answers
69 views
sed command or awk commands to remove text
I have this input file:
target_id length eff_length est_counts tpm
ENST00000583162.1 1066 967 1.69899 1.18376
ENST00000583355.1 891 792 13.8057 11.7445
...
2
votes
2answers
35 views
Changing the date format within a CSV file
I have a program that outputs data in the format
date, time,field1,field2,field3,fieldn
12/20/14,08:01:53,318.622,0.93712,21.6867,1.1089
the file has many columns which all need to stay the same
...
0
votes
5answers
73 views
Replace substring of characters with awk and sed
I have a file which contains a very long string of characters and I would like to replace a substring of it with Ns. Example:
test
ABCDABCDABCD
I would like to replace a substring of it with all ...
0
votes
3answers
33 views
Replace and increment a line in bash
How can I read, increment and replace a value in a file?
foo="val"
ver="1.2.0001"
...
Now I would like to increment the "0001" to 0002".
2
votes
2answers
117 views
Why does my script create an extra character?
I know this isn't a good thing to do and/or a good solution but I was just messing around trying to answer this question and I came across the following problem.
The goal is to read the input file ...
0
votes
2answers
50 views
Replace character based on range of coordinates
Hi I have a text file with thousands of lines which looks like this:
NNNNN
NNNNN
NNNNN
NNNNN
I would like to substitute based on their location coordinate all the letters within a specific range to ...
0
votes
2answers
33 views
Replacing all characters in text file with specific character
I have a file which looks like:
file.txt
NNNP
PPZL
ZZNN
I would like to replace all the letters to "N" nd have an output like this with the same number of lines as the input:
(my file here has 3 ...
1
vote
1answer
37 views
How to read a file with two columns into a script so that each line read as two variables
I have a file that has two columns like such.
cat test.txt
100 2007
FFF 1993
7E7 1994
4BB 1995
I need to input each of the lines into a script, and have the two columns assigned as separate ...
1
vote
2answers
34 views
extract starting and ending with a special character
I have a comma separated file which is not a standard csv file like this:
XYZ,143,ABC/genomes/date/pa341,dog,available
567,FTR/genomes/date/ha76870,horse,waiting
214,GEN/genomes/date/btr256,N/A,...
0
votes
2answers
54 views
How replace tabs with comma after 3rd tab position in a file
I have tab separated text file, the number of fields is not the same for every record. But I want to replace all tabs with commas after 3rd tab in the file.
Input
100 Apple COOK USA EUROPE CHINA
200 ...
0
votes
3answers
55 views
how to select columns in a file by the column name in a txt file in unix
I have a big file with ~ 1000 columns. The first 2 columns are chromosome and SNP (single nucleotide polymorphism)position, the rest are samples.
Here is the first few line and rows:
#CHROM POS ...
2
votes
2answers
34 views
append names of multiple .txt files and rename concatenation destination file
I am new to bash shell scripting, apologies if this was already asked.
I have combinations of multiple files such as:
USA.txt Florida.txt Miami.txt
I would like to join those files and create a new ...
0
votes
3answers
56 views
Swapping words with sed
I need help with sed, actually with Linux regular expressions:
I need to change two parameters between parentheses, for instance, if the line is
mysqli_select_db($par1, $par2)<br/>
I need to ...
0
votes
2answers
29 views
get all the lines from second file matching with values of a specific column in the first file
I have two files:
first file:
ab34,ax,55,gt
ab46,56,thy,ix
ab34,tg,47,xy
at87,kl,89,th
ab46,ut,87,er
second file:
x1,ab34,78,got
wq,at87,100,op
90,ab46,la,gf
expected result:
ab34,ax,55,gt,x1,...
0
votes
5answers
55 views
Putting a command inside awk to merge columns of different files
I have a file.txt for example (it does not have the same number of columns for each row):
1 2 3 4
5 5 6
7 7 7 7 9 10
I have another file (file2.txt) that contains 2 columns
a b
c d
e f
...
1
vote
1answer
28 views
Search through line for a string and print that line only once to another file
I have a text file that has unique lines by timestamps. This below is the example of the lines in that text file:
111.111.11.11: Jun 18 08:01:34 2017: abc\john.doe1 Windows...
111.111.11.11: Jun 18 ...
2
votes
2answers
173 views
Using Sed to add counter to keyword
Suppose I have a file like this.
Keyword "name"
aaa bbb
ccc ddd
Keyword "another name"
eee fff
ggg
hhh iii
and so on.
I want to change the keyword lines in the file to have counters starting from ...
0
votes
1answer
16 views
How to insert comment at specific variable line number by sed command? [duplicate]
I want to comment out a line (add a # to the beginning). I used this command: sed -i '96s/^/#&/' file.txt on Linux to insert comment at line # 96.
If the line number is a variable, then how can I ...
0
votes
3answers
117 views
Invert the last byte of a file?
I'd like to test my hashing program by inverting a single byte at the end of a file without modifying the timestamp. Keeping the timestamp from changing is easy enough, but how do I use a unix utility ...
1
vote
2answers
29 views
how to add new column with constant value in a text file
I have a large text file with several columns and want to add a new column (no matter where) with constant value of 10000
please help
-2
votes
2answers
42 views
Perl::Find a substring and display a whole word
I have a text file that has some data like:
PADHOGOA1 IOP055_VINREG5_1 ( .IO(VINREG5_1), .MONI(),
.MON_D(px_IOP055_VINREG5_1_MON_D), .R0T(px_IOP054_VINREG5_0_R0T),
.IO1() );
PADV30MA0 ...
0
votes
0answers
32 views
removing duplicates [duplicate]
I have a large text file with many rows but some of the rows have duplicates, however only in the first column:
A B
A C
E F
E G
I want to remove the duplicate lines and it doesn't matter which ...
0
votes
1answer
27 views
Replacing Line if it Matches Line from Other File in Nested [While, IF, Sed] Statement
I have a 17k lines-long file F1 with 8 fields per line formatted as follows:
id1 field2 field3 field4 field5 field6 field7 field8
id2 field2 field3 field4 field5 field6 field7 field8
The second file ...
1
vote
2answers
42 views
Delete last characters of strings in a txt file
I have a file called TempsModel.txt which is created as follows:
ls /media/Elise/2811226E69F71131/ModelOutput/* > TempsModel.txt
so it lists all the files in that directory. These files are all ...
1
vote
3answers
104 views
Perl: Inserting at nth position in file
I have content in my file :
123456789
My output is:
1234CC789
Desired Output is:
1234CC56789
I am insert PP at nth position suppose.
But when I am inserting any text it is inserting the text at ...
2
votes
2answers
71 views
Re-order lines and merge others based on a specific criteria
A weak point in my cli foo is awk. I could probably solve the following with elaborate scripting, but I'm pretty sure awk is the best tool for the job and for the life of me I can't figure out the ...
-3
votes
0answers
36 views
How do I cat only the nth line to mth line? [duplicate]
Say I have a large file with a millionth of lines but I only want to cat the line 100 to 500,000 how do I achieve that?
0
votes
1answer
24 views
Use reverse i search to cycle through only commands matching pattern
The contents of the command history file using history | less is:
555 ls
556 ls -a
557 echo "hello"
558 echo "hello again"
559 cd
560 pwd
561 echo "hello hello"
562 ls
563 echo "hello hello hello"
...
0
votes
1answer
14 views
ffprobe run command for multiple videos
I am currently running this command for each video file ending in .mp4 in directory /home/videos to output the running time of each one in /root/videoduration.txt
ffprobe -v error -show_entries ...
1
vote
2answers
34 views
swap text before and after a character [closed]
I have a large text file code.txt with the following equalities
oReservist.Role = Sheets("RECAP").Cells(ligne, 2)
oReservist.Surname = Sheets("RECAP").Cells(ligne, 3)
...
0
votes
2answers
49 views
How to Aggregate and get group count in specific column from text file by perl?
I have a big text file(with 15 lines) in this format (srip dip sport dport .....)
I would like to group distinct first column and get the group counts,
input file:
10.10.10.1 10.10.66.150 60761 502 ...
0
votes
2answers
22 views
Combine text file into one line removing tabs and double commars (DKIM Key)
I am able to combine all lines into one using the following
cat input.txt | tr -d '\n' > output.txt
But as you can see below my data is separated by tabs and double commars.
mail._domainkey IN ...
