Manipulation of text by programs, scripts, etc.

learn more… | top users | synonyms

1
vote
1answer
318 views

Shell script and crontab to capture file size, name and date in a CSV file and then send via email?

I have a number of zip files in a certain folder. I want to capture file name and the size of the files in a CSV format in day wise and put it in crontab so I can report on a daily basis, and then ...
14
votes
14answers
5k views

How to read an IP address backwards?

If we have this string (IP address): 192.168.1.1 How can I derive the (DNS reverse record form) from this string, so it will be shown like 1.1.168.192.in-addr.arpa using a shell script?
0
votes
1answer
130 views

Remove given parts of a string with sed?

$ echo '<a href="mailto:NA?Subject=AB42525216 - FOOBAR bla bla - bla">NA</a>' <a href="mailto:NA?Subject=AB42525216 - FOOBAR bla bla - bla">NA</a> $ echo '<a ...
5
votes
4answers
853 views

how can we use awk to get multiple words after a significant word?

I want to connect to my DB only once to do multiple SELECT queries as follows: #!/bin/bash #Begin Code query=$(echo "SELECT COUNT(*) from USER_IPTable; SELECT User from USER_IPTable; ...
1
vote
4answers
94 views

Getting data under or above a line

This is my file: print energy 1 end Local net energy (PE 0) = 1.635393034217E+05 kJ/mol Global net ELEC energy = 1.635393034217E+05 kJ/mol print energy 2 end Local net energy (PE 0) = ...
2
votes
3answers
924 views

Merging multiple tab-delimited tables based on common columns

I am new to the field and I want to compare and merge 500 hundred tab-delimited files, each with two columns ("Position" and "Poly"). Examples of file1, and file2 (and other files) and the final ...
4
votes
6answers
750 views

Extract lines based on words on position, not column

I have a file like this: v ATOM 57 O LYS A 7 2.254 25.484 18.942 1.00 14.46 ATOM 77 NH1AARG A 8 5.557 19.204 13.388 0.55 24.50 TER 1648 ...
6
votes
3answers
1k views

How to have command history with timestamps output to the terminal continuously?

I use a simple alias to enable the "tracking" of commands in one or many terminal window(s): alias trackmi='export PROMPT_COMMAND="history -a; $PROMPT_COMMAND"' Then I just tail -f my .bash_history ...
6
votes
10answers
6k views

Copy only Specific text of a file to another

I have a file abc.txt the contents are <classpathentry kind="src" path="Sources"/> <classpathentry kind="con" path="WOFramework/ERExtensions"/> <classpathentry kind="con" ...
2
votes
2answers
395 views

Compare two files and return a true or false value

I have two files with one column. If the first rows in the two files are same, then print 1 otherwise print 0. file1: M A A M file2: M M A A output: 1 0 1 0 Can anybody show me an awk script ...
0
votes
3answers
90 views

Compare three file and print the highest value

I have three files. All of them having only one column. File1 M 0 M A File2 A 0 M A File3 M 0 A A compare each row in the three files, i.e, for first row (M,A,M) hence the maximum occurrence is ...
0
votes
4answers
732 views

compare columns in a one file and print the count of matches

I have a CSV file that contains two columns. I want to compare the two columns and print the number of matches. file1 M,M A,M A,A M,A Expected output: If the first columns in the first row is the ...
2
votes
3answers
348 views

conditional coloring in grep

I have a simple bash script. One line of my script matches a "pattern" with grep and colors the match grep -i --color=always -- "$1" I need to modify it, so that it only colors the match, if the ...
4
votes
6answers
7k views

Regex that would grep numbers after specific string

So I have a line: ID: 54376 Can you help me make a regex that would only return numbers without "ID:"? NOTE: This string is in a file.
2
votes
1answer
85 views

Substitute values from file1 to file2 awk

I have an input file in xml FILE1 : <Sector sectorNumber="1"> <Cell cellNumber="1" cellCreated="YES" cellIdentity="" cellRange="35000" numberOfTxBranches="1" hsCodeResourceId="0"/> ...
1
vote
1answer
2k views

Finding all files containing a word and then counting the number of lines

I need to recursively find all files that contain a specific word and if the word exists in file I need to find out the number of lines in that file. I have been trying to use grep but I have been not ...
0
votes
4answers
154 views

Print text between tags (inclusive) if certain text is found

I've got a task to extract data from several Apache servers. The task is to print out: <Directory ...> ... </Directory> where +ExecCGI is located within. Let me give an example to ...
0
votes
1answer
76 views

skip line that doesn't contain a particular character eg .colon

I am reading lines from a file in a while-loop. while read line do #process line done I want to skip lines that do not contain a colon.
4
votes
4answers
300 views

Wrap sentences into columns

I'm looking for a way to take sentences and wrap them. Using something like column is not what I want as in my scenario there will be multiple sentence columns in the 80 characters. Is there any ...
0
votes
4answers
8k views

extract value between two search patterns on same line

I have the following in a file Output.dat. I need to extract the value between dn: uid= and ,ou= dn: uid=user1,ou=Active,ou=Member,dc=domain,dc=org dn: ...
1
vote
3answers
275 views

How to remove string between two strings?

INPUT: $ cat a.txt FOO<td align="right" style='mso-number-format:"\[$-409\]m\/d\/yy\\ h\:mm\\ AM\/PM\;\@";' x:str>BAR OUTPUT: $ sed 's/SOMEMAGIC//g' a.txt FOOBAR My question: How can I ...
3
votes
10answers
3k views

Finding the maximum of the values in a file

I have a file which contains several numbers, as follows. 21 12 33 35 21 12 33 44 52 63 14 12 23 34 11 12 13 53 1 12 43 33 44 How can I find the maximum of these values, which is 63? I wanted to ...
3
votes
5answers
130 views

Wrap lines to a specified number of fields

I have a text file which contains several vectors, as follows. The components of these vectors are separated with space and they are spanned in a few lines. This file is generated after I run a ...
8
votes
6answers
3k views

How can I get a specific line from a file? [duplicate]

I want to extract an exact line from a very big file. For example, line 8000 would be gotten like this: command -line 8000 > output_line_8000.txt
1
vote
2answers
233 views

Create file with specific formatting from directory contents

I am using Solaris and csh. In $home/test there are five files: a.txt, b.txt, c.txt, d.txt and e.txt. If I run % find $home/test/ -type f -print > ../filenames.txt that file will look like: ...
1
vote
3answers
10k views

Delete lines from a file using awk or sed based on the variable from previous command

I have a CSV file (say temp.csv) with strings and numbers. I need to delete the lines which contain a particular pattern of string at the beginning of the lines in the file. For example, here is my ...
8
votes
3answers
3k views

Remove only the commas present within the double quotes

In a text file, I want to remove , (commas) and also the " (quotes) (only if the double quotes contains numbers seperated by commas). 56,72,"12,34,54",x,y,"foo,a,b,bar" Expected ouput ...
1
vote
3answers
79 views

File comparing and appending based on fields

I want to compare 2 files, locus_file.txt is a very large file and atrr.txt is a small file. I want to match the first 2 columns of the first file to the second column of atrr.txt and print the ...
2
votes
1answer
200 views

cut slice of file based on time when lines are written

i have a process running always with logging file could reach up to 5 giga, sometime i have to take slice of this log based on interval of time (usually i used dd). is there any command in linux to ...
1
vote
2answers
216 views

How to mark dates which are in the past?

From the answers I updated the question: Now, it's: $ date "+%F %H:%M" 2014-06-04 11:26 This is the input: $ cat input.txt <tr><td>2014-04-24 14:01</td><td>2014-06-04 ...
1
vote
1answer
112 views

How to export string about France to Microsoft excel file?

One string file Str.rc about French language is here: ID_STR_BRIGHTNESS;,"Luminosité" ID_STR_CHILE_EASTER_ISLAND;,"Île de Pâques" ID_STR_CURRENT_CH;,"Saisie chaîne" ID_STR_DETAILS;,"Détails" ...... ...
6
votes
6answers
475 views

How can I avoid escaped delimiter characters in a Unix command?

I need to take the sum of a field which is tilde (~) delimited. Problem I have is that my data is also having delimiter escaped. Example 1~CEO~ashok\~kumar~1000 As we see in 3rd field above we ...
15
votes
2answers
12k views

Simplest way to comment/uncomment certain lines using command line

Is there a way to comment/uncomment a shell/config/ruby script using command line? for example: $ comment 14-18 bla.conf $ uncomment 14-18 bla.conf this would add or remove # sign on bla.conf on ...
2
votes
1answer
143 views

How to convert .doc to ODF with a command line?

I'd like to convert many Microsoft Word documents to ODF (the OO.org formata with a command line (because there are many of them). (I need them for reading afterwards on a device where reading OO.org ...
1
vote
2answers
308 views

How can I format a flat file using awk?

Below is the format of some table data contained within a single file: ;NULL;ABCD;ABHJARS;;ABCD;ABCD;Y;;;;;;;;;A; ;NULL;XEU-ANKD;XEU-AJKD;;ABCD;ABCD;Y;;;;;;;;;A; . . ;11744;AMKDIONSKH;AMKDJ AN DJ ...
2
votes
4answers
408 views

Take a sum based upon particular columns

I have a huge pile of data—around 30 to 40 Million data items. We need to process these files and send to another interface team. Below is my file format which we receive c1 c2 c3 c4 c5 ...
2
votes
2answers
368 views

Sed? Stripping all formatting, i.e line breaks and whitespaces, from a report text file, while masking out certain pieces

I am working on a project in which I need to remove all formatting from a text file including whitespaces and line breaks, then replace any colons with pipes. I've made some headway but I cannot find ...
7
votes
5answers
569 views

List out strings which are substrings of other strings in the list

I have a list of names like so: dog_bone dog_collar dragon cool_dragon lion lion_trainer dog I need to extract out names that appear in other names like so: dragon lion dog I looked through the ...
2
votes
3answers
321 views

Remove spaces and headers from a dumped database table

I have dumped the database table data into a flat file and below is how the data looks like : (Kindly copy from below ;metier_code ; ;-------------------------; (0 rows affected) ...
2
votes
2answers
169 views

How to remove the prefix from an e-mail in a table?

this is the INPUT: $ echo -e ...
4
votes
7answers
376 views

How can I groups numbers in a file

I have a file with numbers in float format. I can review them via sort -rn numbers.txt | less I would like though to be able to "group" them. I.e. easily see how many are in the same range. To give an ...
4
votes
2answers
2k views

How can I extract the numbers in the file using sed or any other tool?

I have a file that has this format [ 2014/05/01 10:48:26 | 13963 | DEBUG ] It took 11.16837501525879 seconds to complete the process So I have thousands of lines like this and I would like ...
1
vote
1answer
162 views

Parse/Manipulate in awk

I am new to awk programming. I have a question on manipulating text file, which is required to draw certain network based images in a visualization software (Circos http://circos.ca) I have input ...
0
votes
0answers
301 views

Editing metadata with bash

I have a bunch of photos(jpeg, bmp, png) that I have arranged in such a manner that the metadata is in html form in a separate file. I would like create some script to read this html file and edit ...
2
votes
1answer
94 views

Text processing for words connected by hyphen

I have text files (*.txt) which I process, I want to find words connected by hyphen such as "drug-associated" or "drug-linked", "drug-related". I need your help to convert the first one to ...
0
votes
1answer
62 views

Python: Move lines backward in a textfile

Imagine a text file which contains random text and two unique markers 01 text text text 02 text text text 03 __DELETE_THIS_FIRST__ 04 text text text 05 text text text 06 text text text 07 text text ...
0
votes
4answers
435 views

How to extract the similarities between two strings

I have two strings in two different sentences: string 1: 30 mutation alanine for valine string 2: alanine at position 30 is there away to find the similarities between them as it is clear they ...
3
votes
4answers
451 views

Color matching but print other lines too [duplicate]

I have an alias: alias grep='grep --color=auto'. I wish to have some way to print not only matching row but all of them and just color matching patterns. I doubt is there an option to do that in the ...
1
vote
2answers
2k views

Issues Splitting CSV files

Im trying to split a csv file into multiple files using the below command..This command splits into 5 files but the data is truncated in one of the file? How do I fix this issue? >>split -n 5 ...
10
votes
9answers
1k views

How can I print the longest number in a string?

I am looking for a method to print the longest number in a string. E.g.: If I have the string 212334123434test233 how can I print 212334123434 ? Note: I am looking for the longest continuous ...