Manipulation of text by programs, scripts, etc.

learn more… | top users | synonyms

1
vote
3answers
78 views

Merging multiple tab-delimited tables based on common columns

I am new to the field and I want to compare and merge 500 hundred tab-delimited files, each with two columns ("Position" and "Poly"). Examples of file1, and file2 (and other files) and the final ...
4
votes
6answers
238 views

Extract lines based on words on position, not column

I have a file like this: v ATOM 57 O LYS A 7 2.254 25.484 18.942 1.00 14.46 ATOM 77 NH1AARG A 8 5.557 19.204 13.388 0.55 24.50 TER 1648 ...
5
votes
3answers
224 views

How to have command history with timestamps output to the terminal continuously?

I use a simple alias to enable the "tracking" of commands in one or many terminal window(s): alias trackmi='export PROMPT_COMMAND="history -a; $PROMPT_COMMAND"' Then I just tail -f my .bash_history ...
6
votes
10answers
519 views

Copy only Specific text of a file to another

I have a file abc.txt the contents are <classpathentry kind="src" path="Sources"/> <classpathentry kind="con" path="WOFramework/ERExtensions"/> <classpathentry kind="con" ...
1
vote
2answers
137 views

Compare two file and return a true or false value

I have two files with one column. If the first rows in the two files are same the print 1 otherwise print 0. file1: M A A M file2: M M A A output: 1 0 1 0 Can anybody tell me the awk script? ...
0
votes
3answers
60 views

Compare three file and print the highest value

I have three files. All of them having only one column. File1 M 0 M A File2 A 0 M A File3 M 0 A A compare each row in the three files, i.e, for first row (M,A,M) hence the maximum occurrence is ...
0
votes
4answers
45 views

compare columns in a one file and print the count of matches

I have a CSV file that contains two columns. I want to compare the two columns and print the number of matches. file1 M,M A,M A,A M,A Expected output: If the first columns in the first row is the ...
4
votes
3answers
143 views

conditional coloring in grep

I have a simple bash script. One line of my script matches a "pattern" with grep and colors the match grep -i --color=always -- "$1" I need to modify it, so that it only colors the match, if the ...
2
votes
6answers
368 views

Regex that would grep numbers after specific string

So I have a line: ID: 54376 Can you help me make a regex that would only return numbers without "ID:"? NOTE: This string is in a file.
2
votes
1answer
58 views

Substite values from file1 to file2 awk

I have input file in xml FILE1 : <Sector sectorNumber="1"> <Cell cellNumber="1" cellCreated="YES" cellIdentity="" cellRange="35000" numberOfTxBranches="1" hsCodeResourceId="0"/> ...
1
vote
1answer
336 views

Finding all files containing a word and then counting the number of lines

I need to recursively find all files that contain a specific word and if the word exists in file I need to find out the number of lines in that file. I have been trying to use grep but I have been not ...
0
votes
4answers
62 views

Print text between tags (inclusive) if certain text is found

I've got a task to extract data from several Apache servers. The task is to print out: <Directory ...> ... </Directory> where +ExecCGI is located within. Let me give an example to ...
0
votes
1answer
38 views

skip line that doesn't contain perticular character eg .colon

I am reading lines from the file in while loop. while read line do #process line done I want to skip the line that does not contain colon. Any help?
4
votes
4answers
152 views

Wrap sentences into columns

I'm looking for a way to take sentences and wrap them. Using something like column is not what I want as in my scenario there will be multiple sentence columns in the 80 characters. Is there any ...
0
votes
4answers
120 views

extract value between two search patterns on same line

I have the following in a file Output.dat. I need to extract the value between dn: uid= and ,ou= dn: uid=user1,ou=Active,ou=Member,dc=domain,dc=org dn: ...
1
vote
3answers
51 views

How to remove string between two strings?

INPUT: $ cat a.txt FOO<td align="right" style='mso-number-format:"\[$-409\]m\/d\/yy\\ h\:mm\\ AM\/PM\;\@";' x:str>BAR OUTPUT: $ sed 's/SOMEMAGIC//g' a.txt FOOBAR My question: How can I ...
3
votes
7answers
497 views

Finding the maximum of the values in a file

I have a file which contains several numbers, as follows. 21 12 33 35 21 12 33 44 52 63 14 12 23 34 11 12 13 53 1 12 43 33 44 How can I find the maximum of these values, which is 63? I wanted to ...
3
votes
5answers
111 views

Wrap lines to a specified number of fields

I have a text file which contains several vectors, as follows. The components of these vectors are separated with space and they are spanned in a few lines. This file is generated after I run a ...
7
votes
6answers
489 views

How can I get a specific line from a file? [duplicate]

I want to extract an exact line from a very big file. For example, line 8000 would be gotten like this: command -line 8000 > output_line_8000.txt
1
vote
2answers
78 views

Create file with specific formatting from directory contents

I am using Solaris and csh. In $home/test there are five files: a.txt, b.txt, c.txt, d.txt and e.txt. If I run % find $home/test/ -type f -print > ../filenames.txt that file will look like: ...
1
vote
3answers
619 views

Delete lines from a file using awk or sed based on the variable from previous command

I have a CSV file (say temp.csv) with strings and numbers. I need to delete the lines which contain a particular pattern of string at the beginning of the lines in the file. For example, here is my ...
6
votes
3answers
359 views

Remove only the commas present within the double quotes

In a text file, I want to remove , (commas) and also the " (quotes) (only if the double quotes contains numbers seperated by commas). 56,72,"12,34,54",x,y,"foo,a,b,bar" Expected ouput ...
1
vote
3answers
58 views

File comparing and appending based on fields

I want to compare 2 files, locus_file.txt is a very large file and atrr.txt is a small file. I want to match the first 2 columns of the first file to the second column of atrr.txt and print the ...
2
votes
1answer
70 views

cut slice of file based on time when lines are written

i have a process running always with logging file could reach up to 5 giga, sometime i have to take slice of this log based on interval of time (usually i used dd). is there any command in linux to ...
1
vote
2answers
83 views

How to mark dates which are in the past?

From the answers I updated the question: Now, it's: $ date "+%F %H:%M" 2014-06-04 11:26 This is the input: $ cat input.txt <tr><td>2014-04-24 14:01</td><td>2014-06-04 ...
1
vote
1answer
57 views

How to export string about France to Microsoft excel file?

One string file Str.rc about French language is here: ID_STR_BRIGHTNESS;,"Luminosité" ID_STR_CHILE_EASTER_ISLAND;,"Île de Pâques" ID_STR_CURRENT_CH;,"Saisie chaîne" ID_STR_DETAILS;,"Détails" ...... ...
6
votes
6answers
77 views

How can I avoid escaped delimiter characters in a Unix command?

I need to take the sum of a field which is tilde (~) delimited. Problem I have is that my data is also having delimiter escaped. Example 1~CEO~ashok\~kumar~1000 As we see in 3rd field above we ...
10
votes
2answers
811 views

Simplest way to comment/uncomment certain lines using command line

Is there a way to comment/uncomment a shell/config/ruby script using command line? for example: $ comment 14-18 bla.conf $ uncomment 14-18 bla.conf this would add or remove # sign on bla.conf on ...
1
vote
1answer
46 views

How to convert .doc to ODF with a command line?

I'd like to convert many Microsoft Word documents to ODF (the OO.org formata with a command line (because there are many of them). (I need them for reading afterwards on a device where reading OO.org ...
1
vote
2answers
91 views

How can I format a flat file using awk?

Below is the format of some table data contained within a single file: ;NULL;ABCD;ABHJARS;;ABCD;ABCD;Y;;;;;;;;;A; ;NULL;XEU-ANKD;XEU-AJKD;;ABCD;ABCD;Y;;;;;;;;;A; . . ;11744;AMKDIONSKH;AMKDJ AN DJ ...
2
votes
4answers
71 views

Take a sum based upon particular columns

I am having a huge pile of data around 30 to 40 Million data. We need to process these files and send to another interface team. Below is my file format which we recevie c1 c2 c3 c4 c5 c6 A B ...
2
votes
2answers
74 views

Sed? Stripping all formatting, i.e line breaks and whitespaces, from a report text file, while masking out certain pieces

I am working on a project in which I need to remove all formatting from a text file including whitespaces and line breaks, then replace any colons with pipes. I've made some headway but I cannot find ...
7
votes
5answers
391 views

List out strings which are substrings of other strings in the list

I have a list of names like so: dog_bone dog_collar dragon cool_dragon lion lion_trainer dog I need to extract out names that appear in other names like so: dragon lion dog I looked through the ...
2
votes
3answers
201 views

Remove spaces and headers from a dumped database table

I have dumped the database table data into a flat file and below is how the data looks like : (Kindly copy from below ;metier_code ; ;-------------------------; (0 rows affected) ...
2
votes
2answers
86 views

How to remove the prefix from an e-mail in a table?

this is the INPUT: $ echo -e ...
4
votes
7answers
318 views

How can I groups numbers in a file

I have a file with numbers in float format. I can review them via sort -rn numbers.txt | less I would like though to be able to "group" them. I.e. easily see how many are in the same range. To give an ...
2
votes
2answers
156 views

How can I extract the numbers in the file using sed or any other tool?

I have a file that has this format [ 2014/05/01 10:48:26 | 13963 | DEBUG ] It took 11.16837501525879 seconds to complete the process So I have thousands of lines like this and I would like ...
1
vote
1answer
116 views

Parse/Manipulate in awk

I am new to awk programming. I have a question on manipulating text file, which is required to draw certain network based images in a visualization software (Circos http://circos.ca) I have input ...
0
votes
0answers
69 views

Editing metadata with bash

I have a bunch of photos(jpeg, bmp, png) that I have arranged in such a manner that the metadata is in html form in a separate file. I would like create some script to read this html file and edit ...
2
votes
1answer
48 views

Text processing for words connected by hyphen

I have text files (*.txt) which I process, I want to find words connected by hyphen such as "drug-associated" or "drug-linked", "drug-related". I need your help to convert the first one to ...
0
votes
1answer
24 views

Python: Move lines backward in a textfile

Imagine a text file which contains random text and two unique markers 01 text text text 02 text text text 03 __DELETE_THIS_FIRST__ 04 text text text 05 text text text 06 text text text 07 text text ...
0
votes
4answers
76 views

How to extract the similarities between two strings

I have two strings in two different sentences: string 1: 30 mutation alanine for valine string 2: alanine at position 30 is there away to find the similarities between them as it is clear they ...
2
votes
4answers
121 views

Color matching but print other lines too [duplicate]

I have an alias: alias grep='grep --color=auto'. I wish to have some way to print not only matching row but all of them and just color matching patterns. I doubt is there an option to do that in the ...
1
vote
2answers
106 views

Issues Splitting CSV files

Im trying to split a csv file into multiple files using the below command..This command splits into 5 files but the data is truncated in one of the file? How do I fix this issue? >>split -n 5 ...
10
votes
9answers
1k views

How can I print the longest number in a string?

I am looking for a method to print the longest number in a string. E.g.: If I have the string 212334123434test233 how can I print 212334123434 ? Note: I am looking for the longest continuous ...
6
votes
4answers
633 views

We need a “C” list that contains all the elements of “A”, but doesn't have any of “B”

We have two lists. A bigger "A": A=`echo -e '1\n2\n3\n4\n5'` echo "$A" 1 2 3 4 5 and a smaller "B": B=`echo -e '1\n2\n3'` echo "$B" 1 2 3 Q: But we need a third list that contains all the ...
2
votes
3answers
45 views

way to select lines that show variations

I have a huge file that contains 80 genomes and five chromosomes. I was wondering is there is an easy way to select only the lines from ONE chromosome (for example 1) but only at the position showing ...
3
votes
3answers
131 views

Subsetting a file's contents based on value in a column

I have a text file as (for example): apple V$NFY_Q6_rc V=0.741 apple V$HOXA7_01_rc V=0.454 apple V$ALPHACP1_01_rc V=0.695 orange V$SP4_Q5 V=0.747 grapes V$SP1_Q2_01 V=0.677 grapes ...
1
vote
2answers
138 views

Remove just one word before pattern match in shell script

Input: int foo(int a, void * b, int c); Output: int foo(int, void *, int); Want to use "," as pattern to match. If "," pattern is matched then remove the previous word i.e. a, b & c.
1
vote
1answer
99 views

bash scripts - remove duplicate rows with smaller value

Here is an example file: 1 5 20 40 60 2 1 20 20 20 2 2 30 30 30 4 5 40 40 40 7 2 50 30 30 7 1 20 20 20 I would like to remove duplicate rows (with the same value in the first column) with a bigger ...