1
vote
3answers
53 views

Extract part of string using grep

I have 3 files in a directory MYO144064T MYO144064TA MYO144064TX Where the digits and 11th character will change. In my csh script, I want to extract the letters 'A' and 'X' probably using grep ...
4
votes
4answers
62 views

How to switch the first letters of two words?

$ echo 'foo bar' | sed SOMEMAGIC boo far $ echo 'one two' | sed SOMEMAGIC tne owo Q: in general, how can I replace the "b" from the "bar" to the "f" of the "foo"? And in reverse too, replace the "f" ...
1
vote
3answers
125 views

Print the last-but-one field

How to print the word before the last word in line (with ksh or awk or sed or perl one liner) Example 1: echo one two three will print "two" Example 2: echo 1 2 3 4 5 6 will print "5"
0
votes
2answers
49 views

search for first pattern and copy the string inside the ( ) into the 3rd field in lines where the second pattern found

Here are my spec: first search for "#firstpattern(" copy the content inside the ( ) to the 3rd field inside the ( ) of all lines where the "secondpattern" found. remove the entire line with the ...
3
votes
2answers
110 views

case sensitive substitution; same target ids

I am struggled myself to make a case sensitive replacement in a text file. Please find below a segment of my sed file that I am running as sed -f file.sed < input.txt > output.txt ...
-1
votes
1answer
45 views

Can the string “abcedf” be matched to string “bafcde” in a single line command?

I am planning to implement an indexing structure in my program. For example, if I have 100 rows in the table, I will number these rows from 1 to 100 in another column by appending an _ to the end of ...
1
vote
3answers
70 views

Pattern Matching and Delete the whole line

I want to delete all the Lines of File 1, if Column1 of File1 matches exactly with Column 1 File2. File 1: r001:21:10 21 AAAAAATTTGC * = XM:21 r002:21:10 21 YAAAATTTGC * ...
4
votes
2answers
140 views

Replace string with sequential index

Can someone suggest an elegant way to accomplish this? Input: test instant () test instant () ... test instant () //total 1000 lines output should be: test instant1 () test ...
5
votes
2answers
300 views

Remove unknown unicode character from textfiles - sed, other bash/shell methods

I need to search and replace all occurences of an unknown character in some files having the same name. Opening such files with vi, I read <91> code for that character. Opening them with nano, I ...
1
vote
1answer
37 views

pcregrep fails due to segmentation fault

I am extracting the HTML source of a web page using the below command. curl http://www.linkedin.com/pub/ramesh-venkataraman/63/621/b42 -s | w3m -dump -T text/html > foo.txt The foo.txt has the ...
3
votes
3answers
158 views

extract only the substring after double quotes - grep

I have a file which is as below. <a href="http://firstlink.com" title="title1"> <a href="http://secondlink.com" title="title2"> <a href="http://thirdlink.com" title="title3"> <a ...
3
votes
1answer
44 views

Reorder Multiple Line Blocks with Sed

I am trying to reorder generated output as follows; given a tex fragment as follows, I want to move the /^Constant/ down below /Dummies/, with the line before and after /^Constant/. Further, I would ...
-2
votes
3answers
71 views

Deleting Lines, which match a particular Identifier from another file

I have 2 files. File 1, has an identifier (eg. D7MHBF:11:1449:1988) and every new entry starts with @. It has few more fields, which are not important in our analysis. File 2 consists of a column of ...
6
votes
1answer
156 views

How can I replace a string in a file(s)?

Replacing strings in files based on certain search criteria is a very common task. How can I replace string foo with bar in all files in the current directory? do the same recursively for sub ...
0
votes
2answers
48 views

Turning percentages into decimals?

I've got a CSV file that has percentages here and there expressed in text format, e.g. 173.91,0.00%,36.10%,1.71 Question of the day is how do I turn that example into 173.91,0,0.3610,1.71 Edit ...
1
vote
1answer
82 views

Parsing csv with sed and perl one-liner?

I have a web server which stores configurations to PostgreSQL, and I plan to retrieve information from a certain table and then later parse it into csv format. PAYLOAD <?xml version="1.0" ...
1
vote
4answers
65 views

Print the string between two parentheses

I have file with these lines G8 = P(G1,G3) G9 = P(G3,G4) G12 = P(G2,G9) G15 = P(G9,G5) G16 = P(G8,G12) G17 = P(G12,G15) I need the output as G1,G3 G3,G4 ..... How can I do it with sed/grep ...
0
votes
1answer
33 views

rearranging from colum to row in a csv [duplicate]

I have a CSV file that presently has an output like this 1 2 3 4 5 what I need to do is reconfigure the output as follows 1 2 3 4 5 all the entry data on one line, any help would be greatly ...
0
votes
1answer
76 views

best programming language for I/O intensive tasks [closed]

I need to process "big data" (nearly 1TB which can be partitioned into smaller chunks). Consider several programming languages (Java, Python, C++, Perl) and other scripting options (Perl, awk, sed, ...
4
votes
6answers
96 views

Only show the lines that are in all textfiles at least once

a.txt cat a.txt a b x c b.txt cat b.txt d e a f the q: SOMEMAGICK *.txt a Q: how can I show only the lines that are in all *.txt files?
3
votes
3answers
1k views

importing data from a text file to a bash script

I need to import a text file like this below: AMBI CBNK CCXI CERE CLACW CNYD DAEG DLIA DLLR DNDN DSGX HAST HIBB, HPOL IRDMZ MARK NEPT NFEC NINE NRCIB OMER PLCC PLPM PSUN UNTK There are 25 entries, ...
1
vote
3answers
269 views

How to limit MAX an MIN length of a string?

"nothing" means no OUTPUT. Q: how can I limit the MIN and MAX length of a piped STRING? (MIN: x>=3 chars, MAX: x<=32 chars) INPUT: echo 12 | MAGIC OUTPUT: nothing INPUT: echo 123 | ...
1
vote
1answer
50 views

How to make URL's out of this text with least resources?

INPUT: nyeeeeeeeeeeeeeee.txt:sadfsadf sadjflsajfd:jasdlfjalsfd foo/asdfsafd.txt:laksjdlfajsfd OUTPUT: <a href="nyeeeeeeeeeeeeeee.txt">nyeeeeeeeeeeeeeee.txt</a>:sadfsadf <a ...
5
votes
1answer
152 views

Execute a command within 'sed'

I have a list of random memory addresses saved in a text file, and I need to sed them out, decode them, and save them back to my file. However, the decoding operation I'm trying to perform is handled ...
2
votes
6answers
1k views

Repeat each line multiple times

Would like to have each line in a file repeated a fixed number of times. e.g. have each line repeated four times: a b c becomes: a a a a b b b b c c c c I've done some searching, and there are ...
2
votes
2answers
125 views

linux & solaris - separate netmask IP's from ordinary IP's

I have file with IP address and netmask IP's my target is to cut the netmask IP's from the file_with_IPs.txt and paste them to another file as file_with_only_netmask_ips.txt remark - netmask IP can ...
-3
votes
4answers
356 views

perl one liner + how to replace any last IP address octets

How to replace any last IP address octets (four octet ) to 0 remark: Implementation should be by Perl one linear on linux machines for example 192.9.200.124 will replaced to 192.9.200.0 ...
2
votes
3answers
457 views

I want to find duplicates in a file and add a character to the end of the line on the 1st match

I am trying to find duplicates in a file and once a match is found mark the 1st match with a character or word on the end of the line. eg my file (test.html) contains the following entries host= ...
2
votes
3answers
1k views

Easy way to parse syslog date format

I'd like to write a script that analyses the last week's syslog. But my syslog logs in a stupid date format, e.g. "May 22". If it logged as 2013-05-22, this task would be trivial. Is there a way ...
3
votes
3answers
135 views

sed + how to remove character/s that start or ended on each number

How do I remove the . character(s) that start in the beginning of each number or end on each number? Remark – perl one liner also good alternative for sed. Example input: .23.12.44.5. .233.3.3.3 ...
6
votes
4answers
1k views

How to remove multiple blank lines from a file?

I have some text-files I use to take notes in - just plain text, usually just using cat >> file. Occasionally I use a blank line or two (just return - the new-line character) to specify a new ...
1
vote
2answers
55 views

Trimming one part if equal to another part?

I use an e-mail quote attribution string like this: On dd mmm yyyy hh:mm +hhmm, from info@example.org (Example Dot Org): where info@example.org is the sender's e-mail address and Example Dot Org is ...
0
votes
2answers
160 views

Reordering strings in linux

How can I change lines around using the command line? e.g. I have these lines: Acct-Status-Type = Start User-Name = "37XXXXXXX" Event-Timestamp = "Apr 12 2013 15:56:55 AMT" I need to ...
3
votes
3answers
949 views

Delete XML node containing certain element

I want to remove all Placemarks from a KML file that contain the element <tessellate>. The following block should be wholly removed: <Placemark> ...
0
votes
3answers
81 views

Grep end regex matching

On OS X, system_profiler SPHardwareDataType outputs: Hardware Overview: Model Name: MacBook Pro Total Number of Cores: 4 L2 Cache (per Core): 256 KB L3 Cache: 8 MB Memory: 8 GB I want to ...
2
votes
3answers
250 views

awk, sed, grep, perl… which to print out in this case?

I have this syntax in a file (http response times from analog): <thead><tr><th class="x">seconds</th><th class="R">reqs</th><th ...
1
vote
3answers
439 views

linux/solaris + verify duplicate valid IP address from file

what the best way to find duplicate IP from file ( I have ksh script in this script I need to write function that check for duplicate IP ) for example if IP - 192.1926.23.52 exists twice in file - ...
1
vote
5answers
347 views

Replacing lines in files with file contents

I have several files which contain some PHP includes and I want to substitute them with the file contents. The file looks like foo <?php include("file1.php"); ?> bar baz <?php ...
1
vote
1answer
516 views

Linux + replace STRING/WORD in file according to rule

My target is to replace any word/string/Any characters in file with new word/string according to the following rules: If numeric characters on the left side of word/string then we not replace the ...
2
votes
4answers
673 views

Replace all but a set of characters in a file with newline

how to delete all characters in file except numbers and "." , each word (numbers/dot) should be in new line in file see example2 the solution can be with sed or awk or ksh syntax remark - ...
0
votes
3answers
71 views

Fixing header and print

I have header starting with '>' and I want fix the header by keeping first word and removing other shown as in output.txt and print it input.txt >AGAJ01065549.1 ...
0
votes
3answers
634 views

Split and print

I have tab-demlimted text file with 50,000lines. I want to split the first column and print as shown in output.txt Input.txt rt|371443144|mb|MN556661.1| 2200443 A rt|371443344|mb|MN556645.1| ...
5
votes
2answers
1k views

Replacing missing value blank space with zero

I have input.txt tab-delimited text file around 30K lines, I would like to check each row (s1..s30K lines) for missing value (i.e blank space) and fill the missing value with zero value.See out.txt ...
1
vote
1answer
192 views

Combining files with 4 columns each, into 4 files for each column [closed]

So I have a bunch of text files which look like this: T1 r1 a b c d r2 d e f g ... T2 r1 h i j k r2 l m n o ... And I want to make FOUR different text files in the following ...
1
vote
5answers
190 views

frequency of occurence

I have a tab-delimited text file, that is around 3K lines long. I want to calculate frequency of data appearing in it, in 10,000 ranges. Input.txt Price 500 1500 10001 15000 17000 ...
2
votes
4answers
186 views

Output some sections of a text file [closed]

I have input like the following: >gi|481443199|gb|MH447771.1| 49 0 0 0 1 50 0 0 1 0 >gi|987078004|gb|RTYJ01067061.1| 412 0 0 1 ...
2
votes
3answers
965 views

Average rows with same first column

Given a file with two columns: Id ht 510 69 510 67 510 65 5l0 62 510 59 601 29 601 26 601 21 601 20 I need a way to coalesce all rows with the same ID into one that has an average height. In this ...
2
votes
3answers
3k views

Extracting column from comma separated text

I have a long comma-separated delimited file with 20K lines. Here's a sample: "","id","number1","number2","number3","number4","number5","number6","number7" ...
2
votes
3answers
107 views

Need to parse a double entry table with two pairs

I have this two-entry table resulting from a program output: Variable1 : Value1 Variable2 : Value2 Variable3 : Value3 Variable4 : Value4 Variable5 : Value5 Variable6 : Value6 ...
0
votes
2answers
194 views

Getting data from matching list

list.txt GETID_17049_knownids_1/2_Confidence_0.625_Length_2532 GETID_9248_knownids_6/10_Confidence_0.439_Length_2474 GETID_11084_knownids_3/3_Confidence_0.600_Length_1451 ...