Manipulation of text by programs, scripts, etc.

learn more… | top users | synonyms

6
votes
4answers
187 views

remove lines that is newer than the given date in a file

I'm stuck on how I can go about deleting lines that are newer than given date. Here is a snippet of contents of a file. buildsave.txt 647919 2013/11/30 647946 2013/11/30 647955 2013/12/01 648266 ...
1
vote
3answers
70 views

Remove characters in pattern of words

I have sentences with words that have a pattern such as p.G345L and p.X31Z. I need to remove p. so I get G345L and X31Z.
4
votes
3answers
317 views

Counting the occurrences of the string

As input, we have some ids which are strings. How do we calculate what identifier has appeared most frequently and how many times? Example Input: cat dog cat dog dog dog dog fly spider spider ...
0
votes
3answers
259 views

compare two files based on a column and print it

I have two big files of 400,000 lines. I want to compare the column 1 of the second file with column 1 of first file recursively. If they match I would like to print the whole line. It is a sorted ...
2
votes
1answer
91 views

How to replace a leading zero with white space in SED?

Trying to find how to replace a leading zero with white space in SED. Attempting to get day of month and replace with space to filter some data that I am screen scraping. The code I have is as ...
2
votes
2answers
111 views

how to parse a file that is changed continuously using awk?

for example if we have a file contains : Hello world Hello earth Hi everybody we can easily parse it to get the first and second columns of the lines contains the word Hello using awk: awk ...
2
votes
1answer
58 views

Remove all the new lines in a text file using sed [duplicate]

I would like to remove all the new lines in my text file with sed, I use this code: sed -i-o 's/\n//g' mytext.txt but nothing changes. I must be missing something... That is the content of my ...
4
votes
2answers
59 views

Using perl to modify a LaTeX document

I am trying to use perl in order to modify a LaTeX document. In fact, I am trying to change the citation format from for example: . \cite{poelma20113d} to \cite{poelma20113d}. In other words, I ...
1
vote
1answer
135 views

Shell script and crontab to capture file size, name and date in a CSV file and then send via email?

I have a number of zip files in a certain folder. I want to capture file name and the size of the files in a CSV format in day wise and put it in crontab so I can report on a daily basis, and then ...
11
votes
13answers
2k views

How to read an IP address backwards?

If we have this string (IP address): 192.168.1.1 How can I derive the (DNS reverse record form) from this string, so it will be shown like 1.1.168.192.in-addr.arpa using a shell script?
0
votes
1answer
36 views

Remove given parts of a string with sed?

$ echo '<a href="mailto:NA?Subject=AB42525216 - FOOBAR bla bla - bla">NA</a>' <a href="mailto:NA?Subject=AB42525216 - FOOBAR bla bla - bla">NA</a> $ echo '<a ...
3
votes
4answers
295 views

how can we use awk to get multiple words after a significant word?

I want to connect to my DB only once to do multiple SELECT queries as follows: #!/bin/bash #Begin Code query=$(echo "SELECT COUNT(*) from USER_IPTable; SELECT User from USER_IPTable; ...
1
vote
4answers
87 views

Getting data under or above a line

This is my file: print energy 1 end Local net energy (PE 0) = 1.635393034217E+05 kJ/mol Global net ELEC energy = 1.635393034217E+05 kJ/mol print energy 2 end Local net energy (PE 0) = ...
1
vote
3answers
90 views

Merging multiple tab-delimited tables based on common columns

I am new to the field and I want to compare and merge 500 hundred tab-delimited files, each with two columns ("Position" and "Poly"). Examples of file1, and file2 (and other files) and the final ...
4
votes
6answers
241 views

Extract lines based on words on position, not column

I have a file like this: v ATOM 57 O LYS A 7 2.254 25.484 18.942 1.00 14.46 ATOM 77 NH1AARG A 8 5.557 19.204 13.388 0.55 24.50 TER 1648 ...
5
votes
3answers
233 views

How to have command history with timestamps output to the terminal continuously?

I use a simple alias to enable the "tracking" of commands in one or many terminal window(s): alias trackmi='export PROMPT_COMMAND="history -a; $PROMPT_COMMAND"' Then I just tail -f my .bash_history ...
6
votes
10answers
545 views

Copy only Specific text of a file to another

I have a file abc.txt the contents are <classpathentry kind="src" path="Sources"/> <classpathentry kind="con" path="WOFramework/ERExtensions"/> <classpathentry kind="con" ...
1
vote
2answers
141 views

Compare two file and return a true or false value

I have two files with one column. If the first rows in the two files are same the print 1 otherwise print 0. file1: M A A M file2: M M A A output: 1 0 1 0 Can anybody tell me the awk script? ...
0
votes
3answers
61 views

Compare three file and print the highest value

I have three files. All of them having only one column. File1 M 0 M A File2 A 0 M A File3 M 0 A A compare each row in the three files, i.e, for first row (M,A,M) hence the maximum occurrence is ...
0
votes
4answers
48 views

compare columns in a one file and print the count of matches

I have a CSV file that contains two columns. I want to compare the two columns and print the number of matches. file1 M,M A,M A,A M,A Expected output: If the first columns in the first row is the ...
4
votes
3answers
145 views

conditional coloring in grep

I have a simple bash script. One line of my script matches a "pattern" with grep and colors the match grep -i --color=always -- "$1" I need to modify it, so that it only colors the match, if the ...
2
votes
6answers
402 views

Regex that would grep numbers after specific string

So I have a line: ID: 54376 Can you help me make a regex that would only return numbers without "ID:"? NOTE: This string is in a file.
2
votes
1answer
59 views

Substite values from file1 to file2 awk

I have input file in xml FILE1 : <Sector sectorNumber="1"> <Cell cellNumber="1" cellCreated="YES" cellIdentity="" cellRange="35000" numberOfTxBranches="1" hsCodeResourceId="0"/> ...
1
vote
1answer
361 views

Finding all files containing a word and then counting the number of lines

I need to recursively find all files that contain a specific word and if the word exists in file I need to find out the number of lines in that file. I have been trying to use grep but I have been not ...
0
votes
4answers
64 views

Print text between tags (inclusive) if certain text is found

I've got a task to extract data from several Apache servers. The task is to print out: <Directory ...> ... </Directory> where +ExecCGI is located within. Let me give an example to ...
0
votes
1answer
39 views

skip line that doesn't contain perticular character eg .colon

I am reading lines from the file in while loop. while read line do #process line done I want to skip the line that does not contain colon. Any help?
4
votes
4answers
156 views

Wrap sentences into columns

I'm looking for a way to take sentences and wrap them. Using something like column is not what I want as in my scenario there will be multiple sentence columns in the 80 characters. Is there any ...
0
votes
4answers
130 views

extract value between two search patterns on same line

I have the following in a file Output.dat. I need to extract the value between dn: uid= and ,ou= dn: uid=user1,ou=Active,ou=Member,dc=domain,dc=org dn: ...
1
vote
3answers
54 views

How to remove string between two strings?

INPUT: $ cat a.txt FOO<td align="right" style='mso-number-format:"\[$-409\]m\/d\/yy\\ h\:mm\\ AM\/PM\;\@";' x:str>BAR OUTPUT: $ sed 's/SOMEMAGIC//g' a.txt FOOBAR My question: How can I ...
3
votes
7answers
501 views

Finding the maximum of the values in a file

I have a file which contains several numbers, as follows. 21 12 33 35 21 12 33 44 52 63 14 12 23 34 11 12 13 53 1 12 43 33 44 How can I find the maximum of these values, which is 63? I wanted to ...
3
votes
5answers
111 views

Wrap lines to a specified number of fields

I have a text file which contains several vectors, as follows. The components of these vectors are separated with space and they are spanned in a few lines. This file is generated after I run a ...
7
votes
6answers
497 views

How can I get a specific line from a file? [duplicate]

I want to extract an exact line from a very big file. For example, line 8000 would be gotten like this: command -line 8000 > output_line_8000.txt
1
vote
2answers
80 views

Create file with specific formatting from directory contents

I am using Solaris and csh. In $home/test there are five files: a.txt, b.txt, c.txt, d.txt and e.txt. If I run % find $home/test/ -type f -print > ../filenames.txt that file will look like: ...
1
vote
3answers
733 views

Delete lines from a file using awk or sed based on the variable from previous command

I have a CSV file (say temp.csv) with strings and numbers. I need to delete the lines which contain a particular pattern of string at the beginning of the lines in the file. For example, here is my ...
6
votes
3answers
386 views

Remove only the commas present within the double quotes

In a text file, I want to remove , (commas) and also the " (quotes) (only if the double quotes contains numbers seperated by commas). 56,72,"12,34,54",x,y,"foo,a,b,bar" Expected ouput ...
1
vote
3answers
58 views

File comparing and appending based on fields

I want to compare 2 files, locus_file.txt is a very large file and atrr.txt is a small file. I want to match the first 2 columns of the first file to the second column of atrr.txt and print the ...
2
votes
1answer
73 views

cut slice of file based on time when lines are written

i have a process running always with logging file could reach up to 5 giga, sometime i have to take slice of this log based on interval of time (usually i used dd). is there any command in linux to ...
1
vote
2answers
85 views

How to mark dates which are in the past?

From the answers I updated the question: Now, it's: $ date "+%F %H:%M" 2014-06-04 11:26 This is the input: $ cat input.txt <tr><td>2014-04-24 14:01</td><td>2014-06-04 ...
1
vote
1answer
58 views

How to export string about France to Microsoft excel file?

One string file Str.rc about French language is here: ID_STR_BRIGHTNESS;,"Luminosité" ID_STR_CHILE_EASTER_ISLAND;,"Île de Pâques" ID_STR_CURRENT_CH;,"Saisie chaîne" ID_STR_DETAILS;,"Détails" ...... ...
6
votes
6answers
81 views

How can I avoid escaped delimiter characters in a Unix command?

I need to take the sum of a field which is tilde (~) delimited. Problem I have is that my data is also having delimiter escaped. Example 1~CEO~ashok\~kumar~1000 As we see in 3rd field above we ...
10
votes
2answers
879 views

Simplest way to comment/uncomment certain lines using command line

Is there a way to comment/uncomment a shell/config/ruby script using command line? for example: $ comment 14-18 bla.conf $ uncomment 14-18 bla.conf this would add or remove # sign on bla.conf on ...
1
vote
1answer
50 views

How to convert .doc to ODF with a command line?

I'd like to convert many Microsoft Word documents to ODF (the OO.org formata with a command line (because there are many of them). (I need them for reading afterwards on a device where reading OO.org ...
1
vote
2answers
96 views

How can I format a flat file using awk?

Below is the format of some table data contained within a single file: ;NULL;ABCD;ABHJARS;;ABCD;ABCD;Y;;;;;;;;;A; ;NULL;XEU-ANKD;XEU-AJKD;;ABCD;ABCD;Y;;;;;;;;;A; . . ;11744;AMKDIONSKH;AMKDJ AN DJ ...
2
votes
4answers
71 views

Take a sum based upon particular columns

I am having a huge pile of data around 30 to 40 Million data. We need to process these files and send to another interface team. Below is my file format which we recevie c1 c2 c3 c4 c5 c6 A B ...
2
votes
2answers
79 views

Sed? Stripping all formatting, i.e line breaks and whitespaces, from a report text file, while masking out certain pieces

I am working on a project in which I need to remove all formatting from a text file including whitespaces and line breaks, then replace any colons with pipes. I've made some headway but I cannot find ...
7
votes
5answers
393 views

List out strings which are substrings of other strings in the list

I have a list of names like so: dog_bone dog_collar dragon cool_dragon lion lion_trainer dog I need to extract out names that appear in other names like so: dragon lion dog I looked through the ...
2
votes
3answers
210 views

Remove spaces and headers from a dumped database table

I have dumped the database table data into a flat file and below is how the data looks like : (Kindly copy from below ;metier_code ; ;-------------------------; (0 rows affected) ...
2
votes
2answers
89 views

How to remove the prefix from an e-mail in a table?

this is the INPUT: $ echo -e ...
4
votes
7answers
321 views

How can I groups numbers in a file

I have a file with numbers in float format. I can review them via sort -rn numbers.txt | less I would like though to be able to "group" them. I.e. easily see how many are in the same range. To give an ...
2
votes
2answers
157 views

How can I extract the numbers in the file using sed or any other tool?

I have a file that has this format [ 2014/05/01 10:48:26 | 13963 | DEBUG ] It took 11.16837501525879 seconds to complete the process So I have thousands of lines like this and I would like ...