Manipulation of text by programs, scripts, etc.

learn more… | top users | synonyms

3
votes
6answers
206 views

Remove everything between second and second last occurance of match

I have string which I would like to format. I would like to remove everything between second ; and second last ;. Input cellular ...
0
votes
1answer
56 views

How to get `pdftotext` to output text in a readable encoding?

I converted a PDF file into a txt file using pdftotext. As an example, i have the sentence "This is the first study on the functional relevance of", notice the f in "first", when i process this sentence ...
3
votes
2answers
206 views

Nested 'awk' in a 'while' loop, parse two files line by line and compare column values

I need some help with a combination of awk & while loop. I have two simple files with columns (normal ones are very large), one representing simple intervals for an ID=10(of coding ...
1
vote
4answers
143 views

SED command to format text file

I need to create a SED command to modify an input file with some conditions explained below. I have an input file something like below. List item Rad# ; ID_KEY ; UNIT_ID 1 ; 30000000004 ; 8417920 ...
2
votes
2answers
50 views

Pipe Viewer - Progress monitor performance consequence

I am writing a batch script to sort through gigs and gigs of data. All of the data is text but the script will take a long long time to execute. I would like to give some visual indication that the ...
0
votes
3answers
40 views

Replacing line with its line number

How can I replace the whole line with no matching pattern just with the number of the line? sed -i "${line}p c\ "$PWD" " file.txt
0
votes
2answers
140 views

sed command to replace a string from one file with entire contents of another file [duplicate]

I want to write a sed (or awk) command to replace a string from one file with the entire contents of another file. Note that second file from which I want to get the content has more than one line. I ...
5
votes
2answers
395 views

BASH base conversion from decimal to hex

In Bash, how does one do base conversion from decimal to another base, especially hex. It seems easy to go the other way: $ echo $((16#55)) 85 With a web-search, I found a script that does the ...
5
votes
2answers
235 views

How to count duplicated last columns without removing them?

I have a file that contains 4 columns. I want to compare the last three columns and count how many times they occur without deleting any of the lines. I just want the count to be present in front of ...
2
votes
3answers
92 views

deduplication of lines in a large file

The size of the file is 962,120,335 bytes. HP-UX ******B.11.31 U ia64 ****** unlimited-user license hostname> what /usr/bin/awk /usr/bin/awk: main.c $Date: 2009/02/17 15:25:17 $Revision: ...
2
votes
1answer
41 views

Updating config files within the script that references them?

I have no problem adding config files to a given script, but automatically updating them is another matter. In this instance I'm providing a few variables to a script, and I'd like the script to be ...
1
vote
3answers
61 views

Add parentheses around numbers

I have a file filled with numbers: 1 6 8 9 10 11 I want to enclose each number in parentheses so basically have the final output as: (1) (6) (8) (9) (10) (11) and so on.
5
votes
8answers
861 views

How to put sentences on separate lines on Linux

I have an assignment to put sentences in a text file on separate lines. Something like this almost works: cat file.txt | tr '.' '\n' But I don't want to lose dots, question marks and exclamation ...
4
votes
3answers
217 views

sum columns based on matching fields

I have encountered a problem that exceeds my basic unix knowledge and would really appreciate some help. I have a large file in the following format: 2 1019 0 12 2 1019 3 0 2 1021 0 2 ...
3
votes
1answer
109 views

how to change the replacement of word in the specific line using sed command?

I have to change a word in the 4th line so I used this command: sed -i '4 s/previous_word/new_word/' filename and it worked perfectly. But if I save the line number in a variable and try the same ...
2
votes
2answers
60 views

vim: using a for-loop-iterator-variable in a search pattern

I am trying to use a for loop iterator variable in a vim search pattern to determine for a range of words how man times they occur in a file. What I do so far is: for i in range(1,40) | %s/SiTg//gn | ...
0
votes
3answers
85 views

Removing duplicate entries in text file

I am doing some sorting with uniq -d command. it's not getting me the desired results. My file is like: >TCONS_00000066 174 206 33 >TCONS_00000066 210 281 72 >TCONS_00000066 227 ...
1
vote
2answers
136 views

Replace string in the first column

I have a file which looks like: >TCONS_00000066_1 [3 - 170] >TCONS_00000066_2 [174 - 206] >TCONS_00000066_3 [2 - 223] >TCONS_00000066_4 [210 - 281] ...
2
votes
2answers
62 views

Using the sed command in Unix

how do you take one word from a file and change it to an absolute path? Basically I need to take one word from a unix file and change it to a absolute directory path in that file. The file name is ...
2
votes
1answer
134 views

Transform .csv into 3 columns and a row

Sample Input: id,Product1,Product2,Product3,Product4 1,0.1,0.3,0.8,0.7 2,0.6,0.7,0.5,0.9 I need output as : id,productname,product_val 1,Product1,0.1 1,Product2,0.3 1,Product3,0.8 1,Product4,0.7 ...
0
votes
1answer
52 views

Compare text files with grep

Lets say I have two files, I want to find goodbye which I can do with grep but what i'm struggling with is once I find goodbye matching the next line with the first column of file 1. File1 sss aa ...
0
votes
2answers
25 views

How to use variables in substitution with vim regular expressions

I have a big file that contains among others serveral lines of the following form: USet07-1 USet07-2 USet08-1 USet08-2 . . . USet22-2 . . . I want to remove the hyphen/dash - from these strings in ...
1
vote
3answers
174 views

how to delete line from the XML file

how to delete only the line from xml file ( with sed/awk or perl one liner line ) that start with: <units> and ended with </Networks> as the follwoing <units><unit ip= ...
1
vote
2answers
45 views

How to only leave in given characters in a file?

Purpose: tr -cd '\12\40-\176' but with accents, ex.: á, í, ű, ő, ö, ü, ű, ú, ó, é. In octal: "á" is 341 "\n" is 12 So I tried: [user@notebook ~]$ printf 'XXXá\nXXX' | tr -cd '\12\341' | ...
2
votes
2answers
62 views

Find out on which line in text file is matching word

Is there any way to find out on which line of the text document is some word which is matching pattern for example with grep or something. Thanks.
0
votes
1answer
106 views

Read only specific lines from text file and save matches into variables

I've got this problem. I need to extract some words from text file, which I find by specific pattern in that text. For example: /this/is/path: file1.txt,date-3 file2.txt,date-2 /this/is/path2: ...
2
votes
3answers
94 views

How to cut away the end of a textfile after a specified line

I have == Changelog == in a textfile and I want to create a new file where this line and everything after it will be deleted. I am sure the answer is here somewhere, I have tried to search, now my ...
1
vote
2answers
64 views

How to read only the second last line of a file

I have a client that updates/writes into myfile.csv arbitrarily. I've written the following code, I left the MySQL snippet out: while true do awk_variables_value_array=`tail -n2 myfile.csv | awk ...
1
vote
1answer
105 views

Bash: Looping through a string

I have a variable Vmac that stores values of multiple MAC addresses, collected from wmic call as follows: Vmac=`wmic --delimiter="," --authentication-file=/path/to/file //IP-ADDR "Select ...
1
vote
2answers
44 views

Some characters disappear after processing a text file

I've a file named addresses.txt : address1@domain.com address2@domain.com address3@domain.com I'd like to echo "address1@domain.com.html" using for loop. Here's what I did : cat addresses.txt | ...
-1
votes
1answer
39 views

Remove line breaks in a FASTA file

I have a fasta file where the sequences are broken up with newlines. I'd like to remove the newlines. Here's an example of my file: >accession1 ATGGCCCATG GGATCCTAGC >accession2 GATATCCATG ...
0
votes
0answers
29 views

match two files [duplicate]

I have 2 files. The first, fileA looks like TCONS_00000066 XLOC_000030 - u q1:XLOC_000030|TCONS_00000066|0|0.000000|0.000000|0.000000|0.000000|- TCONS_00000130 XLOC_000057 - u ...
1
vote
2answers
82 views

match first column of file a with paragraphs of file b

I have 2 files. The first, fileA looks like TCONS_00000066 XLOC_000030 - u q1:XLOC_000030|TCONS_00000066|0|0.000000|0.000000|0.000000|0.000000|- TCONS_00000130 XLOC_000057 - ...
2
votes
3answers
87 views

Remove duplicate entries from a CSV file

I've got a [csv] file with duplicate datum reprinted ie the same data printed twice. I've tried using sort's uniq by sort myfile.csv | uniq -u however there is no change in the myfile.csv, also I've ...
1
vote
1answer
39 views

Replace a string in a text file using a shell script not working properly

I have a text file containing this: 0 0 -1 0 1 0 0 0 0 -1 0 0 1.5 0.0 1.0 1 and I want to replace the 1.0 to 2.0 so that it becomes: 0 0 -1 0 1 0 0 0 0 -1 0 0 1.5 0.0 2.0 1 so I use this ...
2
votes
5answers
182 views

How to reverse the lines between a pattern into another file

I have the following in the input file #Start para 0 hello dq 1 world dq 2 welcomes dq 3 you dq #Start para 0 how tq 1 are tq 2 you tq #Start para 0 say dq 1 hello dq 2 to dq 3 the dq 4 world dq I ...
1
vote
1answer
65 views

Shell : while read line nested

I've two documents: doc1.lst and doc 2.lst I want to take the content of each line and put it as parameters for my SQL query. I tried something like this, please correct me file=doc1.lst ...
3
votes
3answers
173 views

Splitting a sequence into fixed width

I have a file like this which is a two-column tab-separated file. CTGCAGTTTCCCCAAATGTGGGAAACTTGACTGTATAATTTGTGGCAGTGGTA a1 GATTTCCCCAAATGTGGGAAACTCACTCGGCAGGCGTTGATA a2 I want to get an output ...
1
vote
3answers
48 views

How to do find and replace in one field based on the value in the other field?

I have a file like this. IMS00|20140101|Unix|Jan14 IMS01|20140201|Unix|Feb14 IMS02|20140301|Unix|Mar14 IMS03|20140101|Unix|Jan14 IMS04|20140201|Unix|Feb14 My criteria are, if the column 1 value is ...
3
votes
3answers
65 views

Using sed to remove string or paragraph between delimiter

I would like to know who to remove a string or paragraph that are between ((( string ))) Lorem ipsum (((dolor sit amet))), consectetur adipiscing elit. Vestibulum aliquet fringilla est, dictum ...
4
votes
2answers
112 views

Regular expression to match sequences of alphanumerics

Any way using grep/awk/perl to match lines in the form *.* = *.* (i.e. ac4df.bx5dfd8g = ce5def.dd5f7gdgf ) Where * means 0+ alphanumeric char?
1
vote
6answers
316 views

Replace \n with blank in a file

I want to replace \n with blank Input file1 A\n D\n Output file2 I want A D I tried below commands but not able to replace please do needful. cat OUT | tr '\n' ' '
1
vote
2answers
24 views

Transform values in a line by first field

I was wondering if there is an easy way to do the following without writing a script. Transform 1234,"a;b;d" 2345,"e;f;g;h" to 1234,a 1234,b 1234,d 2345,e 2345,f 2345,g 2345,h
1
vote
3answers
135 views

Append based on line count in awk after pattern match

How can I append lines after a pattern match in awk based on how many lines they are after the pattern match. What I have looks like this: Stuff PATTERN a b c a b c a b c ENDOFSECTION Morestuff I ...
2
votes
1answer
28 views

Match and delete lines with ~100 strings

I have a list of roughly 100 entries to be deleted from a csv-delimited file. They are already in another text file called 'tbd.txt; My first thought is to write a bash for loop around 'sed -i' but ...
1
vote
1answer
50 views

Searching file for string surrounded by two other known string?

Question title may be kinda vague, but heres what im attempting to do. I have a html file locally on my server which is changing fairly regularly. It has multiple occurrences of: <p>Jumble of ...
3
votes
1answer
279 views

Bash scripting to scan files for words and create report

Ok yes this is a homework assignment BUT I am NOT looking to have the answers given to me. The scenario is that I need a script that searches all my users home directories for bad words. I need the ...
0
votes
3answers
57 views

How can I capture the common characters between two strings

I'm looking to compare two strings and capture the common part e.g. FILE0000010985.LOG FILE0000010999.LOG FILE0000011000.LOG If I compare 1 to 2, I want the output to be ...
0
votes
3answers
45 views

I want to condense a list of ranges (from-to)

I have data involving codes assigned to ranges in the form of From To Code 0 5 A 5 10 A 10 15 A 15 20 B 20 25 B 25 30 A 30 35 A 35 ...
2
votes
4answers
42 views

grep and print how many times my pattern in file 1 is present in file2

I have a file1 (list of my pattern) like this: file1 Fatty_acid_degradation Aminobenzoate_degradation Amino_sugar_and_nucleotide_sugar_metabolism Amoebiasis and I have a file2 (list of all the ...