Manipulation of text by programs, scripts, etc.

learn more… | top users | synonyms

0
votes
2answers
183 views

How to get the value from rows and columns from text file in bash?

I am working on a bash project. I need to get all the values (one at a time because I will be using them somewhere else) from a text file. The text file I will be getting is structured like this: ...
0
votes
2answers
42 views

Remove a particular line no from a text file

I have .csv file having several lines of data. I have to search for a particular value in the string/record and delete that line itself. eg ; abc.csv col1_id,col2,col3,col4,col5 ...
1
vote
1answer
50 views

Extract time stamps and find the difference

I have a large set of logs as shown below: June 5 15:51:56.606 [3997]: TR#153. State machine - state = CC_STATE_IDLE, event = EVT_ISDN_SETUP June 5 15:51:56.607 [3997]: TR#153): SC ...
3
votes
3answers
194 views

awk filtering live capture with file as pattern list

What I'm trying to do is quite simple. I'm generating output from tshark and redirecting it to awk with a pipe |. Since tshark gets live data, I want that awk search in every output for a pattern ...
1
vote
2answers
91 views

Editing INI-like files with a script

I'm writing a script to automate setting up Puppet agent configuration files in Docker. Basically, I need to ensure that the following section is in /etc/puppet/puppet.conf: [agent] ...
6
votes
8answers
634 views

Grep rest of line…after match

I have a file containing only two lines, with the following structure: $ cat /tmp/pwpower.log 000D6F0000D34227, -114.10 000D6F0001A405C4, -130.09 The values are power values of my solar plant. ...
1
vote
6answers
84 views

Is there a way to find out the number of lines from a command output?

I am trying to compare command outputs between two servers. I want to find out the number of lines from the output of commands such as ls -l. Is there a way to do so? So far I have not found anything. ...
12
votes
5answers
360 views

non-line-oriented tool for string replacement?

I recently asked a question about how to remove the newline character if it occurs after another specific character. Unix text-processing tools are very powerful, but almost all of them deal with ...
13
votes
6answers
761 views

Replace string containing newline in huge file

Anyone know of a non-line-based tool to "binary" search/replace strings in a somewhat memory-efficient way? See this question too. I have a +2GB text file that I would like to process similar to what ...
3
votes
1answer
207 views

Process a file that starts with a BOM (FF FE)

I received a .csv file with the FF FE BOM: $ head -n1 dotan.csv | hd 00000000 ff fe 41 00 64 00 20 00 67 00 72 00 6f 00 75 00 |..A.d. .g.r.o.u.| When using awk to parse it I'm getting a bunch of ...
3
votes
5answers
248 views

How do I extract the content of quoted strings from the output of a command?

I have output from VBoxManage list vms which looks like this: "arch" {de1a1db2-86c5-43e7-a8de-a0031835f7a7} "arch2" {92d8513c-f13e-41b5-97e2-2a6b17d47b67} I need to grab the names arch and ...
4
votes
3answers
184 views

UNIX paste columns and insert zero's for all missing values

I would like to merge specific columns from two txt files containing varying number of rows, but same number of columns (as shown below): file1: xyz desc1 12 uvw desc2 55 pqr desc3 ...
0
votes
7answers
84 views

Extract the number of failures from a test report

How would I parse following output of lein test $ lein test lein test nepleaks-engine.core-test Ran 1 tests containing 1 assertions. 0 failures, 0 errors. Where I want is get number before ...
1
vote
2answers
59 views

Append a column

I've trawled through here and a few other places to try and find an answer to this, but it seems so stupidly simple that no one has answered it. All I need to do is add a single value in a new column ...
3
votes
1answer
143 views

Collapse line vertically so that every word is on a new line

Given a string like: "Hello World, how are you today?" Is there a way in Vim to collapse this line vertically such that every word appears on a new line, i.e.: "Hello World, how are ...
6
votes
3answers
185 views

Shell script to search files for identical text entries

Here's a nice teaser for a shell scripting guru: Take a directory with several text files. Can be a few up to ~1000. All files contain an identifier on a given line (always the same line). Identify ...
2
votes
3answers
243 views

How to replace the content of specific column with awk?

Given there is 40 columns in a record. I want to replace 35th column so that the 35th column will be replace with the content of 35th column and a "$" symbol. What came up my mind is like: awk ...
3
votes
5answers
128 views

Get values block by block in same file

I have a file say "SAMPLE.txt" with following content, P1 10,9:6/123456 P2 blah blah P1 10,9:5/98765 P2 blah blah P1 blah blah P2 I want a output file say "RESULT.txt" as, Value1:123456 ...
0
votes
1answer
64 views

Search a string between two timestamps starting from bottom of file

I was trying to find a string Cannot proceed: the cube has no data in huge test.txt file only between timestamps yesterday 22:30 pm to today 00:30 am. Script: tac test.txt | awk -v today=$(date ...
3
votes
1answer
51 views

comparing columnn values between two unix files

File A Table SAPSR3./1BEA/BBEA_BDH has no BasicFile LOB columns Table SAPSR3./1BEA/BBEA_BDI has no BasicFile LOB columns Table SAPSR3./1BEA/BBEA_DLI has no BasicFile LOB columns Table ...
4
votes
5answers
193 views

Replace new line + 4x space to new line in stdout

I need join 2 lines into one in stdout - replace new line AND 4x space with one space but new line without space don't touch tcpdump -vvv ... -l | xyz this is output: 2014-06-06 AAA BBB ...
1
vote
3answers
574 views

compare two columns of different files and print if it matches

I am using solaris 10 and Grep options involving -f doesnt work . I have two pipe separated files: file1: abc|123|BNY|apple| cab|234|cyx|orange| def|kumar|pki|bird| file 2 : abc|123| ...
6
votes
4answers
207 views

remove lines that is newer than the given date in a file

I'm stuck on how I can go about deleting lines that are newer than given date. Here is a snippet of contents of a file. buildsave.txt 647919 2013/11/30 647946 2013/11/30 647955 2013/12/01 648266 ...
1
vote
3answers
80 views

Remove characters in pattern of words

I have sentences with words that have a pattern such as p.G345L and p.X31Z. I need to remove p. so I get G345L and X31Z.
5
votes
3answers
337 views

Counting the occurrences of the string

As input, we have some ids which are strings. How do we calculate what identifier has appeared most frequently and how many times? Example Input: cat dog cat dog dog dog dog fly spider spider ...
0
votes
3answers
548 views

compare two files based on a column and print it

I have two big files of 400,000 lines. I want to compare the column 1 of the second file with column 1 of first file recursively. If they match I would like to print the whole line. It is a sorted ...
2
votes
1answer
110 views

How to replace a leading zero with white space in SED?

Trying to find how to replace a leading zero with white space in SED. Attempting to get day of month and replace with space to filter some data that I am screen scraping. The code I have is as ...
3
votes
2answers
121 views

how to parse a file that is changed continuously using awk?

for example if we have a file contains : Hello world Hello earth Hi everybody we can easily parse it to get the first and second columns of the lines contains the word Hello using awk: awk ...
3
votes
1answer
74 views

Remove all the new lines in a text file using sed [duplicate]

I would like to remove all the new lines in my text file with sed, I use this code: sed -i-o 's/\n//g' mytext.txt but nothing changes. I must be missing something... That is the content of my ...
4
votes
2answers
62 views

Using perl to modify a LaTeX document

I am trying to use perl in order to modify a LaTeX document. In fact, I am trying to change the citation format from for example: . \cite{poelma20113d} to \cite{poelma20113d}. In other words, I ...
1
vote
1answer
157 views

Shell script and crontab to capture file size, name and date in a CSV file and then send via email?

I have a number of zip files in a certain folder. I want to capture file name and the size of the files in a CSV format in day wise and put it in crontab so I can report on a daily basis, and then ...
13
votes
13answers
2k views

How to read an IP address backwards?

If we have this string (IP address): 192.168.1.1 How can I derive the (DNS reverse record form) from this string, so it will be shown like 1.1.168.192.in-addr.arpa using a shell script?
0
votes
1answer
42 views

Remove given parts of a string with sed?

$ echo '<a href="mailto:NA?Subject=AB42525216 - FOOBAR bla bla - bla">NA</a>' <a href="mailto:NA?Subject=AB42525216 - FOOBAR bla bla - bla">NA</a> $ echo '<a ...
5
votes
4answers
327 views

how can we use awk to get multiple words after a significant word?

I want to connect to my DB only once to do multiple SELECT queries as follows: #!/bin/bash #Begin Code query=$(echo "SELECT COUNT(*) from USER_IPTable; SELECT User from USER_IPTable; ...
1
vote
4answers
89 views

Getting data under or above a line

This is my file: print energy 1 end Local net energy (PE 0) = 1.635393034217E+05 kJ/mol Global net ELEC energy = 1.635393034217E+05 kJ/mol print energy 2 end Local net energy (PE 0) = ...
1
vote
3answers
136 views

Merging multiple tab-delimited tables based on common columns

I am new to the field and I want to compare and merge 500 hundred tab-delimited files, each with two columns ("Position" and "Poly"). Examples of file1, and file2 (and other files) and the final ...
4
votes
6answers
262 views

Extract lines based on words on position, not column

I have a file like this: v ATOM 57 O LYS A 7 2.254 25.484 18.942 1.00 14.46 ATOM 77 NH1AARG A 8 5.557 19.204 13.388 0.55 24.50 TER 1648 ...
5
votes
3answers
252 views

How to have command history with timestamps output to the terminal continuously?

I use a simple alias to enable the "tracking" of commands in one or many terminal window(s): alias trackmi='export PROMPT_COMMAND="history -a; $PROMPT_COMMAND"' Then I just tail -f my .bash_history ...
6
votes
10answers
711 views

Copy only Specific text of a file to another

I have a file abc.txt the contents are <classpathentry kind="src" path="Sources"/> <classpathentry kind="con" path="WOFramework/ERExtensions"/> <classpathentry kind="con" ...
1
vote
2answers
149 views

Compare two file and return a true or false value

I have two files with one column. If the first rows in the two files are same the print 1 otherwise print 0. file1: M A A M file2: M M A A output: 1 0 1 0 Can anybody tell me the awk script? ...
0
votes
3answers
64 views

Compare three file and print the highest value

I have three files. All of them having only one column. File1 M 0 M A File2 A 0 M A File3 M 0 A A compare each row in the three files, i.e, for first row (M,A,M) hence the maximum occurrence is ...
0
votes
4answers
54 views

compare columns in a one file and print the count of matches

I have a CSV file that contains two columns. I want to compare the two columns and print the number of matches. file1 M,M A,M A,A M,A Expected output: If the first columns in the first row is the ...
4
votes
3answers
158 views

conditional coloring in grep

I have a simple bash script. One line of my script matches a "pattern" with grep and colors the match grep -i --color=always -- "$1" I need to modify it, so that it only colors the match, if the ...
2
votes
6answers
706 views

Regex that would grep numbers after specific string

So I have a line: ID: 54376 Can you help me make a regex that would only return numbers without "ID:"? NOTE: This string is in a file.
2
votes
1answer
61 views

Substite values from file1 to file2 awk

I have input file in xml FILE1 : <Sector sectorNumber="1"> <Cell cellNumber="1" cellCreated="YES" cellIdentity="" cellRange="35000" numberOfTxBranches="1" hsCodeResourceId="0"/> ...
1
vote
1answer
534 views

Finding all files containing a word and then counting the number of lines

I need to recursively find all files that contain a specific word and if the word exists in file I need to find out the number of lines in that file. I have been trying to use grep but I have been not ...
0
votes
4answers
69 views

Print text between tags (inclusive) if certain text is found

I've got a task to extract data from several Apache servers. The task is to print out: <Directory ...> ... </Directory> where +ExecCGI is located within. Let me give an example to ...
0
votes
1answer
41 views

skip line that doesn't contain perticular character eg .colon

I am reading lines from the file in while loop. while read line do #process line done I want to skip the line that does not contain colon. Any help?
4
votes
4answers
169 views

Wrap sentences into columns

I'm looking for a way to take sentences and wrap them. Using something like column is not what I want as in my scenario there will be multiple sentence columns in the 80 characters. Is there any ...
0
votes
4answers
229 views

extract value between two search patterns on same line

I have the following in a file Output.dat. I need to extract the value between dn: uid= and ,ou= dn: uid=user1,ou=Active,ou=Member,dc=domain,dc=org dn: ...