0
votes
1answer
10 views

how to remove all double quotes from csv except specific field

I want to remove all double quotes from my csv but not the fourth field (because the four fields represent PATH of file) Please advice how to implement this by sed or awk or perl one liner , etc ...
5
votes
2answers
268 views

How is the inplace modification of a file being done?

What does the "inplace" modification of a file e.g. via sed -i or perl -i mean? My question is about how this inplace modification is done. Is the file copied the modification is done in the copy and ...
1
vote
1answer
38 views

Retrieving fasta sequences using bed file information from locally installed file

I have a .bed file containing around 30000 rows for which I have the sequences retrieved using fetch-sequences module of the rsat tool (http://rsat.ulb.ac.be/rsat/help.fetch-sequences.html#usage). ...
1
vote
4answers
60 views

Remove only given strings from given column?

INPUT: <tr><td>FOOBAAR</td><td>FOOO</td><td>BAAR</td><td><font style=BACKGROUND-COLOR:red>2014-02-14 ...
2
votes
3answers
138 views

replace column of a csv file with column from another file

I have two sample files like this: $ cat file1 abc,sachin cat,kumar $ cat file2 xyz pressure $ cat file3 xyz,sachin pressure, kumar I want the first column of file1 to be replaced with file2. ...
0
votes
4answers
63 views

Print text between tags (inclusive) if certain text is found

I've got a task to extract data from several Apache servers. The task is to print out: <Directory ...> ... </Directory> where +ExecCGI is located within. Let me give an example to ...
0
votes
2answers
54 views

Method to paste specific column when in terminal

I'd like to paste only the third column of data from within the terminal. I originally have something like: 4 SF0086 SF0086 12 SF0087 SF0087 17 TSF141 TSF141 29 TSF070 TSF070 30 TSF028 TSF028 31 ...
4
votes
7answers
319 views

How can I groups numbers in a file

I have a file with numbers in float format. I can review them via sort -rn numbers.txt | less I would like though to be able to "group" them. I.e. easily see how many are in the same range. To give an ...
1
vote
3answers
73 views

Are there any difference using pipes?

Using Ubuntu 12.04. Using pipes (5 times): $ time echo XYZ | sed 's/X/A/g' | sed 's/Y/B/g' | sed 's/Z/C/g' ABC real 0m0.005s user 0m0.000s sys 0m0.000s $ time echo XYZ | sed 's/X/A/g' | sed ...
1
vote
3answers
340 views

Extract part of string using grep

I have 3 files in a directory MYO144064T MYO144064TA MYO144064TX Where the digits and 11th character will change. In my csh script, I want to extract the letters 'A' and 'X' probably using grep ...
4
votes
4answers
107 views

How to switch the first letters of two words?

$ echo 'foo bar' | sed SOMEMAGIC boo far $ echo 'one two' | sed SOMEMAGIC tne owo Q: in general, how can I replace the "b" from the "bar" to the "f" of the "foo"? And in reverse too, replace the "f" ...
1
vote
3answers
239 views

Print the last-but-one field

How to print the word before the last word in line (with ksh or awk or sed or perl one liner) Example 1: echo one two three will print "two" Example 2: echo 1 2 3 4 5 6 will print "5"
0
votes
2answers
68 views

search for first pattern and copy the string inside the ( ) into the 3rd field in lines where the second pattern found

Here are my spec: first search for "#firstpattern(" copy the content inside the ( ) to the 3rd field inside the ( ) of all lines where the "secondpattern" found. remove the entire line with the ...
3
votes
2answers
189 views

case sensitive substitution; same target ids

I am struggled myself to make a case sensitive replacement in a text file. Please find below a segment of my sed file that I am running as sed -f file.sed < input.txt > output.txt ...
2
votes
5answers
239 views

Can the string “abcedf” be matched to string “bafcde” in a single line command?

I am planning to implement an indexing structure in my program. For example, if I have 100 rows in the table, I will number these rows from 1 to 100 in another column by appending an _ to the end of ...
1
vote
4answers
231 views

Pattern Matching and Delete the whole line

I want to delete all the Lines of File 1, if Column1 of File1 matches exactly with Column 1 File2. File 1: r001:21:10 21 AAAAAATTTGC * = XM:21 r002:21:10 21 YAAAATTTGC * ...
4
votes
2answers
337 views

Replace string with sequential index

Can someone suggest an elegant way to accomplish this? Input: test instant () test instant () ... test instant () //total 1000 lines output should be: test instant1 () test ...
5
votes
2answers
991 views

Remove unknown unicode character from textfiles - sed, other bash/shell methods

I need to search and replace all occurences of an unknown character in some files having the same name. Opening such files with vi, I read <91> code for that character. Opening them with nano, I ...
1
vote
1answer
83 views

pcregrep fails due to segmentation fault

I am extracting the HTML source of a web page using the below command. curl http://www.linkedin.com/pub/ramesh-venkataraman/63/621/b42 -s | w3m -dump -T text/html > foo.txt The foo.txt has the ...
3
votes
3answers
967 views

extract only the substring after double quotes - grep

I have a file which is as below. <a href="http://firstlink.com" title="title1"> <a href="http://secondlink.com" title="title2"> <a href="http://thirdlink.com" title="title3"> <a ...
3
votes
1answer
85 views

Reorder Multiple Line Blocks with Sed

I am trying to reorder generated output as follows; given a tex fragment as follows, I want to move the /^Constant/ down below /Dummies/, with the line before and after /^Constant/. Further, I would ...
-2
votes
3answers
84 views

Deleting Lines, which match a particular Identifier from another file

I have 2 files. File 1, has an identifier (eg. D7MHBF:11:1449:1988) and every new entry starts with @. It has few more fields, which are not important in our analysis. File 2 consists of a column of ...
6
votes
1answer
845 views

How can I replace a string in a file(s)?

Replacing strings in files based on certain search criteria is a very common task. How can I replace string foo with bar in all files in the current directory? do the same recursively for sub ...
0
votes
2answers
55 views

Turning percentages into decimals?

I've got a CSV file that has percentages here and there expressed in text format, e.g. 173.91,0.00%,36.10%,1.71 Question of the day is how do I turn that example into 173.91,0,0.3610,1.71 Edit ...
1
vote
1answer
119 views

Parsing csv with sed and perl one-liner?

I have a web server which stores configurations to PostgreSQL, and I plan to retrieve information from a certain table and then later parse it into csv format. PAYLOAD <?xml version="1.0" ...
2
votes
4answers
121 views

Print the string between two parentheses

I have file with these lines G8 = P(G1,G3) G9 = P(G3,G4) G12 = P(G2,G9) G15 = P(G9,G5) G16 = P(G8,G12) G17 = P(G12,G15) I need the output as G1,G3 G3,G4 ..... How can I do it with sed/grep ...
0
votes
1answer
36 views

rearranging from colum to row in a csv [duplicate]

I have a CSV file that presently has an output like this 1 2 3 4 5 what I need to do is reconfigure the output as follows 1 2 3 4 5 all the entry data on one line, any help would be greatly ...
0
votes
1answer
100 views

best programming language for I/O intensive tasks [closed]

I need to process "big data" (nearly 1TB which can be partitioned into smaller chunks). Consider several programming languages (Java, Python, C++, Perl) and other scripting options (Perl, awk, sed, ...
4
votes
6answers
98 views

Only show the lines that are in all textfiles at least once

a.txt cat a.txt a b x c b.txt cat b.txt d e a f the q: SOMEMAGICK *.txt a Q: how can I show only the lines that are in all *.txt files?
3
votes
3answers
3k views

importing data from a text file to a bash script

I need to import a text file like this below: AMBI CBNK CCXI CERE CLACW CNYD DAEG DLIA DLLR DNDN DSGX HAST HIBB, HPOL IRDMZ MARK NEPT NFEC NINE NRCIB OMER PLCC PLPM PSUN UNTK There are 25 entries, ...
1
vote
3answers
394 views

How to limit MAX an MIN length of a string?

"nothing" means no OUTPUT. Q: how can I limit the MIN and MAX length of a piped STRING? (MIN: x>=3 chars, MAX: x<=32 chars) INPUT: echo 12 | MAGIC OUTPUT: nothing INPUT: echo 123 | ...
1
vote
1answer
51 views

How to make URL's out of this text with least resources?

INPUT: nyeeeeeeeeeeeeeee.txt:sadfsadf sadjflsajfd:jasdlfjalsfd foo/asdfsafd.txt:laksjdlfajsfd OUTPUT: <a href="nyeeeeeeeeeeeeeee.txt">nyeeeeeeeeeeeeeee.txt</a>:sadfsadf <a ...
5
votes
1answer
179 views

Execute a command within 'sed'

I have a list of random memory addresses saved in a text file, and I need to sed them out, decode them, and save them back to my file. However, the decoding operation I'm trying to perform is handled ...
1
vote
6answers
2k views

Repeat each line multiple times

Would like to have each line in a file repeated a fixed number of times. e.g. have each line repeated four times: a b c becomes: a a a a b b b b c c c c I've done some searching, and there are ...
2
votes
2answers
144 views

linux & solaris - separate netmask IP's from ordinary IP's

I have file with IP address and netmask IP's my target is to cut the netmask IP's from the file_with_IPs.txt and paste them to another file as file_with_only_netmask_ips.txt remark - netmask IP can ...
-3
votes
4answers
485 views

perl one liner + how to replace any last IP address octets

How to replace any last IP address octets (four octet ) to 0 remark: Implementation should be by Perl one linear on linux machines for example 192.9.200.124 will replaced to 192.9.200.0 ...
2
votes
3answers
744 views

I want to find duplicates in a file and add a character to the end of the line on the 1st match

I am trying to find duplicates in a file and once a match is found mark the 1st match with a character or word on the end of the line. eg my file (test.html) contains the following entries host= ...
2
votes
3answers
1k views

Easy way to parse syslog date format

I'd like to write a script that analyses the last week's syslog. But my syslog logs in a stupid date format, e.g. "May 22". If it logged as 2013-05-22, this task would be trivial. Is there a way ...
3
votes
3answers
153 views

sed + how to remove character/s that start or ended on each number

How do I remove the . character(s) that start in the beginning of each number or end on each number? Remark – perl one liner also good alternative for sed. Example input: .23.12.44.5. .233.3.3.3 ...
6
votes
4answers
2k views

How to remove multiple blank lines from a file?

I have some text-files I use to take notes in - just plain text, usually just using cat >> file. Occasionally I use a blank line or two (just return - the new-line character) to specify a new ...
1
vote
2answers
56 views

Trimming one part if equal to another part?

I use an e-mail quote attribution string like this: On dd mmm yyyy hh:mm +hhmm, from info@example.org (Example Dot Org): where info@example.org is the sender's e-mail address and Example Dot Org is ...
0
votes
2answers
205 views

Reordering strings in linux

How can I change lines around using the command line? e.g. I have these lines: Acct-Status-Type = Start User-Name = "37XXXXXXX" Event-Timestamp = "Apr 12 2013 15:56:55 AMT" I need to ...
3
votes
3answers
1k views

Delete XML node containing certain element

I want to remove all Placemarks from a KML file that contain the element <tessellate>. The following block should be wholly removed: <Placemark> ...
0
votes
3answers
91 views

Grep end regex matching

On OS X, system_profiler SPHardwareDataType outputs: Hardware Overview: Model Name: MacBook Pro Total Number of Cores: 4 L2 Cache (per Core): 256 KB L3 Cache: 8 MB Memory: 8 GB I want to ...
2
votes
3answers
283 views

awk, sed, grep, perl… which to print out in this case?

I have this syntax in a file (http response times from analog): <thead><tr><th class="x">seconds</th><th class="R">reqs</th><th ...
1
vote
3answers
512 views

linux/solaris + verify duplicate valid IP address from file

what the best way to find duplicate IP from file ( I have ksh script in this script I need to write function that check for duplicate IP ) for example if IP - 192.1926.23.52 exists twice in file - ...
2
votes
5answers
434 views

Replacing lines in files with file contents

I have several files which contain some PHP includes and I want to substitute them with the file contents. The file looks like foo <?php include("file1.php"); ?> bar baz <?php ...
1
vote
1answer
552 views

Linux + replace STRING/WORD in file according to rule

My target is to replace any word/string/Any characters in file with new word/string according to the following rules: If numeric characters on the left side of word/string then we not replace the ...
2
votes
4answers
877 views

Replace all but a set of characters in a file with newline

how to delete all characters in file except numbers and "." , each word (numbers/dot) should be in new line in file see example2 the solution can be with sed or awk or ksh syntax remark - ...
0
votes
3answers
71 views

Fixing header and print

I have header starting with '>' and I want fix the header by keeping first word and removing other shown as in output.txt and print it input.txt >AGAJ01065549.1 ...