Questions tagged [text-processing]

Manipulation or examining of text by programs, scripts, etc.

Filter by
Sorted by
Tagged with
1
vote
4answers
41 views

How to use sed get version number

How do I get the version number using sed r-blob-1.0.2-abcsomething urllib3-2.1.4-xyzsomething cfile-8.0-dddsomething How can I use "sed" to cut where there is - and [0-9], but start cutting after ...
0
votes
0answers
32 views

Merge two files searching by key and add at the end

I would merge data from two files into one adding only interesting Values from column. Searching key is UID in column 3 File #1. Also it should remove/omit UID duplicates and empty lines if appear. ...
0
votes
0answers
38 views

How can I remove all cases of a strings on terminal? [on hold]

So I have a dataset with mostly numerical values, but there are some strings categorizing the lines. Example Data: yeet 0 0 0 0 0 0 0 gg2 0 0 ...
5
votes
2answers
64 views

Column deletion based on number of string matches within column

I need a command that will remove any columns in a text file if they have =>${MaxAllowedNumberOfFs} 'F's within the column (a column that will have a varying number of rows). I have some pseudo code ...
1
vote
0answers
28 views

Histogram with FASTA file

I am new to Linux. I have a FASTA file that looks something like this: >scaffold1 AAGACATAATATTTTGGAGGAATTAAAAAATTTAAGATGTATTTTATTATACATGTATTTTATTTATAACATAAATAAACATCCCAAGGAAAAGCAGTAGCT >...
1
vote
1answer
19 views

How to search and replace for multiple words in one shot

I need to search and replace multiple words. For example, in my file, I want to search for any name of week days that appears with a comma after it: Sat, or Sun, or Mon, or Tue, or Wed, or Thur, or ...
0
votes
5answers
54 views

Remove n character after specific word

MY Data: 7468656.xml-87- 7468656.xml-88- 7468656.xml-89- 7468656.xml-90- 7468656.xml-91- 7468656.xml-92- 7468656.xml-93- 7468656.xml:94: 7468656.xml-115- 7468656.xml-116- 7468656.xml-117- 7468656.xml-...
2
votes
4answers
92 views

More elegant approach to append text to a file only if the String doesn't exist?

I want to append NoDisplay=true to a .desktop file, but only if the entry doesn't exist. I do this the following way: grep -q 'NoDisplay=true' '/usr/share/applications/yelp.desktop' || bash -c 'echo "...
0
votes
1answer
68 views

Sed to replace string of 10 characters with 'XXXX ' [duplicate]

I have file which looks like ACCINTCCADVANCESC ACCINTCCADVANCESE ACCINTCCADVANCESW I want to replace INTCCADVAN with XXXX␣␣␣␣␣␣, the ␣s are spaces. Specifically I want to replace a column with more ...
0
votes
5answers
40 views

add specific string at the beginning and at the end of line 1

The input file looks like this H1 O1 C1 H2 H3 C2 H4 H5 O2 C3 H6 H7 C4 H8 H9 O3 C5 H10 H11 C6 H12 ...... I would like to have the following output file z(A) H1 O1 C1 H2 H3 C2 H4 H5 O2 C3 H6 H7 C4 H8 ...
0
votes
2answers
30 views

multiply specific column in a file which consists of thousands columns

The in file looks like this -17.3644 0.00000000 0.00000000 .... -17.2703 0.00000000 0.00000000 .... -17.1761 0.00000000 0.00000000 .... -16.5173 0.00000000 0....
1
vote
3answers
54 views

How to extract domain name from email address and add it to a new column in a pipe delimited file?

I have a pipe-delimited file with an email address in the third column. I need help to create the fourth column out of the domain name from the email address in the third column using awk or sed ...
2
votes
2answers
64 views

Command for converting rows to CSV file

I have a file of the format, with a leading space before each line: "Western Overseas", "Western Overseas", "^", "--", "^", "--", "--", null, 24995, 9977, "CR", "Western Refrigeration ...
0
votes
2answers
29 views

Split delimited string using variable substitution

I have a sample file the lines of which are delimited by "#" and it contains 3 fields. A#B#C D#E#F Can I use UNIX variable substitution instead of the following to get values of each field in the ...
2
votes
2answers
38 views

Split into multiple files by using unique columns in a unix file

I have requirement to split below file (sample.csv) into multiple files by using the unique columns (first 3 are unique columns) sample.csv 123|22|56789|ABCDEF|12AB34|2019-07-10|2019-07-10|443.3400|...
-1
votes
4answers
50 views

Copying first line of text to another text file

Before: file1 123 456 file2 foo testing After: file1 123 456 file2 foo testing 123 Any help would be greatly appreciated.
1
vote
3answers
75 views

Partition a file based on one column

I have the following data set: M1 1 1233 M2 1 3212 M3 1 55323 M4 1 4444233 M5 1 23444 M6 1 555333 M7 1 55567 M8 2 22224 M9 2 55566 M10 2 4567 M11 3 44242 M12 3 234234 ...
0
votes
1answer
49 views

Filtering a .txt file using a regular expression and a second file

I have a file.txt (it does not have the same number of columns for each row): e.g. 1 21:10 21:23 2 1:94 1:100 1:123 3 14:1 14:60 14:23 I have another file (file2.txt) that contains 4 columns (...
0
votes
2answers
69 views

sed command or awk commands to remove text

I have this input file: target_id length eff_length est_counts tpm ENST00000583162.1 1066 967 1.69899 1.18376 ENST00000583355.1 891 792 13.8057 11.7445 ...
2
votes
2answers
35 views

Changing the date format within a CSV file

I have a program that outputs data in the format date, time,field1,field2,field3,fieldn 12/20/14,08:01:53,318.622,0.93712,21.6867,1.1089 the file has many columns which all need to stay the same ...
0
votes
5answers
73 views

Replace substring of characters with awk and sed

I have a file which contains a very long string of characters and I would like to replace a substring of it with Ns. Example: test ABCDABCDABCD I would like to replace a substring of it with all ...
0
votes
3answers
33 views

Replace and increment a line in bash

How can I read, increment and replace a value in a file? foo="val" ver="1.2.0001" ... Now I would like to increment the "0001" to 0002".
2
votes
2answers
117 views

Why does my script create an extra character?

I know this isn't a good thing to do and/or a good solution but I was just messing around trying to answer this question and I came across the following problem. The goal is to read the input file ...
0
votes
2answers
50 views

Replace character based on range of coordinates

Hi I have a text file with thousands of lines which looks like this: NNNNN NNNNN NNNNN NNNNN I would like to substitute based on their location coordinate all the letters within a specific range to ...
0
votes
2answers
33 views

Replacing all characters in text file with specific character

I have a file which looks like: file.txt NNNP PPZL ZZNN I would like to replace all the letters to "N" nd have an output like this with the same number of lines as the input: (my file here has 3 ...
1
vote
1answer
37 views

How to read a file with two columns into a script so that each line read as two variables

I have a file that has two columns like such. cat test.txt 100 2007 FFF 1993 7E7 1994 4BB 1995 I need to input each of the lines into a script, and have the two columns assigned as separate ...
1
vote
2answers
34 views

extract starting and ending with a special character

I have a comma separated file which is not a standard csv file like this: XYZ,143,ABC/genomes/date/pa341,dog,available 567,FTR/genomes/date/ha76870,horse,waiting 214,GEN/genomes/date/btr256,N/A,...
0
votes
2answers
54 views

How replace tabs with comma after 3rd tab position in a file

I have tab separated text file, the number of fields is not the same for every record. But I want to replace all tabs with commas after 3rd tab in the file. Input 100 Apple COOK USA EUROPE CHINA 200 ...
0
votes
3answers
55 views

how to select columns in a file by the column name in a txt file in unix

I have a big file with ~ 1000 columns. The first 2 columns are chromosome and SNP (single nucleotide polymorphism)position, the rest are samples. Here is the first few line and rows: #CHROM POS ...
2
votes
2answers
34 views

append names of multiple .txt files and rename concatenation destination file

I am new to bash shell scripting, apologies if this was already asked. I have combinations of multiple files such as: USA.txt Florida.txt Miami.txt I would like to join those files and create a new ...
0
votes
3answers
56 views

Swapping words with sed

I need help with sed, actually with Linux regular expressions: I need to change two parameters between parentheses, for instance, if the line is mysqli_select_db($par1, $par2)<br/> I need to ...
0
votes
2answers
29 views

get all the lines from second file matching with values of a specific column in the first file

I have two files: first file: ab34,ax,55,gt ab46,56,thy,ix ab34,tg,47,xy at87,kl,89,th ab46,ut,87,er second file: x1,ab34,78,got wq,at87,100,op 90,ab46,la,gf expected result: ab34,ax,55,gt,x1,...
0
votes
5answers
55 views

Putting a command inside awk to merge columns of different files

I have a file.txt for example (it does not have the same number of columns for each row): 1 2 3 4 5 5 6 7 7 7 7 9 10 I have another file (file2.txt) that contains 2 columns a b c d e f ...
1
vote
1answer
28 views

Search through line for a string and print that line only once to another file

I have a text file that has unique lines by timestamps. This below is the example of the lines in that text file: 111.111.11.11: Jun 18 08:01:34 2017: abc\john.doe1 Windows... 111.111.11.11: Jun 18 ...
2
votes
2answers
173 views

Using Sed to add counter to keyword

Suppose I have a file like this. Keyword "name" aaa bbb ccc ddd Keyword "another name" eee fff ggg hhh iii and so on. I want to change the keyword lines in the file to have counters starting from ...
0
votes
1answer
16 views

How to insert comment at specific variable line number by sed command? [duplicate]

I want to comment out a line (add a # to the beginning). I used this command: sed -i '96s/^/#&/' file.txt on Linux to insert comment at line # 96. If the line number is a variable, then how can I ...
0
votes
3answers
117 views

Invert the last byte of a file?

I'd like to test my hashing program by inverting a single byte at the end of a file without modifying the timestamp. Keeping the timestamp from changing is easy enough, but how do I use a unix utility ...
1
vote
2answers
29 views

how to add new column with constant value in a text file

I have a large text file with several columns and want to add a new column (no matter where) with constant value of 10000 please help
-2
votes
2answers
42 views

Perl::Find a substring and display a whole word

I have a text file that has some data like: PADHOGOA1 IOP055_VINREG5_1 ( .IO(VINREG5_1), .MONI(), .MON_D(px_IOP055_VINREG5_1_MON_D), .R0T(px_IOP054_VINREG5_0_R0T), .IO1() ); PADV30MA0 ...
0
votes
0answers
32 views

removing duplicates [duplicate]

I have a large text file with many rows but some of the rows have duplicates, however only in the first column: A B A C E F E G I want to remove the duplicate lines and it doesn't matter which ...
0
votes
1answer
27 views

Replacing Line if it Matches Line from Other File in Nested [While, IF, Sed] Statement

I have a 17k lines-long file F1 with 8 fields per line formatted as follows: id1 field2 field3 field4 field5 field6 field7 field8 id2 field2 field3 field4 field5 field6 field7 field8 The second file ...
1
vote
2answers
42 views

Delete last characters of strings in a txt file

I have a file called TempsModel.txt which is created as follows: ls /media/Elise/2811226E69F71131/ModelOutput/* > TempsModel.txt so it lists all the files in that directory. These files are all ...
1
vote
3answers
104 views

Perl: Inserting at nth position in file

I have content in my file : 123456789 My output is: 1234CC789 Desired Output is: 1234CC56789 I am insert PP at nth position suppose. But when I am inserting any text it is inserting the text at ...
2
votes
2answers
71 views

Re-order lines and merge others based on a specific criteria

A weak point in my cli foo is awk. I could probably solve the following with elaborate scripting, but I'm pretty sure awk is the best tool for the job and for the life of me I can't figure out the ...
-3
votes
0answers
36 views

How do I cat only the nth line to mth line? [duplicate]

Say I have a large file with a millionth of lines but I only want to cat the line 100 to 500,000 how do I achieve that?
0
votes
1answer
24 views

Use reverse i search to cycle through only commands matching pattern

The contents of the command history file using history | less is: 555 ls 556 ls -a 557 echo "hello" 558 echo "hello again" 559 cd 560 pwd 561 echo "hello hello" 562 ls 563 echo "hello hello hello" ...
0
votes
1answer
14 views

ffprobe run command for multiple videos

I am currently running this command for each video file ending in .mp4 in directory /home/videos to output the running time of each one in /root/videoduration.txt ffprobe -v error -show_entries ...
1
vote
2answers
34 views

swap text before and after a character [closed]

I have a large text file code.txt with the following equalities oReservist.Role = Sheets("RECAP").Cells(ligne, 2) oReservist.Surname = Sheets("RECAP").Cells(ligne, 3) ...
0
votes
2answers
49 views

How to Aggregate and get group count in specific column from text file by perl?

I have a big text file(with 15 lines) in this format (srip dip sport dport .....) I would like to group distinct first column and get the group counts, input file: 10.10.10.1 10.10.66.150 60761 502 ...
0
votes
2answers
22 views

Combine text file into one line removing tabs and double commars (DKIM Key)

I am able to combine all lines into one using the following cat input.txt | tr -d '\n' > output.txt But as you can see below my data is separated by tabs and double commars. mail._domainkey IN ...