Manipulation of text by programs, scripts, etc.

learn more… | top users | synonyms

2
votes
1answer
104 views

How to interleave the lines of two or more files?

I have tried to get the 2 files output in a single file. I have tried below command: cat file1 file2 but here file2 data is appending to file1. What I need is file1 1st line followed by file2 ...
2
votes
2answers
126 views

Printing lines with similar text together

I am currently working on analyzing the performance of an application. I've starting by looking at the logs generated by our application and identifying the business logic tasks that are taking more ...
2
votes
1answer
37 views

Extracting lines around keys from a file

I have a large file with around 15 million lines. The odd lines contains the keys (but not only the keys) and the even lines are the data. That is, the file is something like: WRTZ Tyrosine ...
3
votes
3answers
256 views

Sort comma-separated fields on each line by numeric value

I tried this: $ echo "2,3435,1" | sort -n 2,3435,1 $ sort -t',' -n test_sort.txt kill,gill,burger 110,20,30,13 $ cat test_sort.txt 110,20,30,13 kill,gill,burger Why doesn't my sort command work? ...
2
votes
1answer
114 views

Convert xml subtitle file to srt format

I have a xml subtitle that looks like this: <?xml version="1.0" encoding="utf-8" ?><transcript> <Item from="1.16" duration="4.68"><![CDATA[(Dong-hyuk is coming to see you now.) ...
1
vote
1answer
40 views

Retrieving fasta sequences using bed file information from locally installed file

I have a .bed file containing around 30000 rows for which I have the sequences retrieved using fetch-sequences module of the rsat tool (http://rsat.ulb.ac.be/rsat/help.fetch-sequences.html#usage). ...
3
votes
3answers
182 views

String replacement in file

I have the following file: <?xml version="1.0" encoding="utf-8"?> <!--Generated by crowdin.net--> <string name="test" >- test</string> <string name="test" ...
0
votes
2answers
42 views

Insert single successive numbers at the beginning of each line in a text file with Vim

Assume I have a text file containing some strings: string_g string_r string_qqwer string_p string_f string_3qw string_e string_qw And I need to insert increasing or decreasing numbers before the ...
2
votes
1answer
45 views

Change a specific line using sed

I need to change a line in a Java file called DbConnection.java using the sed command. The line is: connection = DriverManager.getConnection("jdbc:mysql://**localhost**/**bajaj**","**root**", ...
5
votes
2answers
197 views

Shell script (bash) to add or replace the license text contained within .c , .h and makefiles?

I have a set of *.c , *.h and Makefiles in a folder where some files contain the license text and some files don't have any license text. So I need a shell script where I can add license text if the ...
2
votes
1answer
30 views

set flag to remove duplicate records from a file based on some values of the record

I have a file having the following records (pipe delimited): SK200-10|ALBUMIN-SK|INS|SEKURE ALBUMIN 2 X ...
5
votes
2answers
66 views

Grepping for a block of text with parts that can be optional

I have multiple entries that describes an event in a very large log file, say A.log. I would like to do two things with the event entries in the log file: Count the number of occurrences of each ...
4
votes
4answers
198 views

Removing lines that are common to the outputs produced by two commands

Say, I have two commands that return some text. For example: $ ./c1 /usr/bin/foo /usr/bin/bar /usr/bin/baz $ ./c2 /usr/bin/foo /usr/bin/qux /usr/bin/buzz /usr/bin/bar I want to remove the duplicate ...
1
vote
1answer
61 views

Fastest way of find and replace in large file with replacing filed present in another file

I have two files. File1 is a csv with 60 fields: 111,Check1|^/h1/h2/h3,22062014184500,20,0..... 111,Check2|^/h43/h40/h9,22062014184500,4,..... 111,Check3|^/h1/h3/h4,22062014184500,0,0,..... File2 ...
1
vote
7answers
84 views

Get text between two special characters in a line using shell

I have a line which comes from a file via: cat aaa.txt | grep "HIJK" Output gives: HIJK="My name is HIJK" I need to get the text My name is HIJK into a variable in a shell script. I have tried ...
3
votes
6answers
68 views

Collapse some repeated parts of successive lines

I have data in notepad like this: 4480-1 4480-2 4480-3 4480-15 4581-1 4581-2 4581-3 4581-4 Can we do it using for loop sort of things? With sed? My required output is ...
0
votes
2answers
35 views

names that occur the largest number of times in a data set

users directory ---------------------- hjsyeh hellodir ; awgshd newdir; hjsyeh hidir; jkdhjk welcomedir the above given are set of users and the directories they created .How can I ...
2
votes
2answers
181 views

How to parse multiple string patterns in linux shell script

I am new to linux shell and I had a hard time thinking of the solution to this. Can somebody help how to parse multiple strings (data) in a file? For below file, I want to parse all the values next to ...
6
votes
3answers
70 views

Print a multiline variable in a column

I have a script that collects values from a cluster. The values, in some cases, have multiple lines. I have a printf format that specifies how the data should be positioned however, it doesn't take ...
1
vote
1answer
42 views

Extract Values from HTML via HTML parser [duplicate]

I initially asked about a way to use grep/awk directly from an HTML file. I got some feedback but I see that people did not have the whole HTML and so the solutions suggested failed. Some have ...
4
votes
2answers
103 views

Replace common values in two files according to first column

How do I replace common values in two files according to first column? I have file 1 (1.txt): CH063_01084 CH063_01083 CH063_13925 CH063_14473 and file 2 (2.txt): CH063_01083 FGSG_16349 4 ...
1
vote
4answers
194 views

Enclose multiple lines in quote Vim

I have blocks of the following form: String that is not supposed to be enclosed in quotes String that is supposed to be enclosed in quotes String that is not supposed to be enclosed in ...
1
vote
1answer
39 views

Join multiple files in a directory column wise

I would like to join about 100 files in the same directory column-wise. I have tried paste file1 file2 | column -s $'\t' -t This works fine for 2 files. But when I try to do it for 100 files, I ...
0
votes
4answers
53 views

More concise methods for wrapping a file than using sed

Introduction script: sed -i '1i <?xml version=\"1.0\" encoding=\"utf-8\"?>' $1 sed -i '/<?xml version=\"1.0\" encoding=\"utf-8\"?>/a<hello>\n\t<world>' $1 sed -i ...
4
votes
4answers
154 views

How to get lines 10 to 100 from a 200 line file into a new file [duplicate]

I have file with 200 lines. I need to extract lines from 10 to 100 and put them into a new file. How do you do this in unix/Linux? What are the possible commands you could use?
6
votes
6answers
245 views

Split large file into chunks without splitting entry

I have a rather large .msg file formatted in the UIEE format. $ wc -l big_db.msg 8726593 big_db.msg Essentially, the file is made up of entries of various length that look something like this: ...
-1
votes
3answers
75 views

Checking if line in file exist

I have a bash script which asks a user for the number of CPU cores and saves it to variable named $cores. Now I want to add this variable to .bashrc, so I ask user how much CPU cores he has and then ...
7
votes
4answers
173 views

Groupwise uniq command?

I am searching for a command to get from a file in this format: hello 32 hello 67 hi 2 ho 1212 ho 1390 ho 3000 To this format (deduplicate by taking the last row of a "group"): hello ...
0
votes
2answers
104 views

How to get the value from rows and columns from text file in bash?

I am working on a bash project. I need to get all the values (one at a time because I will be using them somewhere else) from a text file. The text file I will be getting is structured like this: ...
0
votes
2answers
36 views

Remove a particular line no from a text file

I have .csv file having several lines of data. I have to search for a particular value in the string/record and delete that line itself. eg ; abc.csv col1_id,col2,col3,col4,col5 ...
1
vote
1answer
46 views

Extract time stamps and find the difference

I have a large set of logs as shown below: June 5 15:51:56.606 [3997]: TR#153. State machine - state = CC_STATE_IDLE, event = EVT_ISDN_SETUP June 5 15:51:56.607 [3997]: TR#153): SC ...
3
votes
3answers
155 views

awk filtering live capture with file as pattern list

What I'm trying to do is quite simple. I'm generating output from tshark and redirecting it to awk with a pipe |. Since tshark gets live data, I want that awk search in every output for a pattern ...
1
vote
1answer
60 views

Editing INI-like files with a script

I'm writing a script to automate setting up Puppet agent configuration files in Docker. Basically, I need to ensure that the following section is in /etc/puppet/puppet.conf: [agent] ...
6
votes
8answers
574 views

Grep rest of line…after match

I have a file containing only two lines, with the following structure: $ cat /tmp/pwpower.log 000D6F0000D34227, -114.10 000D6F0001A405C4, -130.09 The values are power values of my solar plant. ...
1
vote
6answers
81 views

Is there a way to find out the number of lines from a command output?

I am trying to compare command outputs between two servers. I want to find out the number of lines from the output of commands such as ls -l. Is there a way to do so? So far I have not found anything. ...
12
votes
5answers
348 views

non-line-oriented tool for string replacement?

I recently asked a question about how to remove the newline character if it occurs after another specific character. Unix text-processing tools are very powerful, but almost all of them deal with ...
13
votes
6answers
592 views

Replace string containing newline in huge file

Anyone know of a non-line-based tool to "binary" search/replace strings in a somewhat memory-efficient way? See this question too. I have a +2GB text file that I would like to process similar to what ...
3
votes
1answer
176 views

Process a file that starts with a BOM (FF FE)

I received a .csv file with the FF FE BOM: $ head -n1 dotan.csv | hd 00000000 ff fe 41 00 64 00 20 00 67 00 72 00 6f 00 75 00 |..A.d. .g.r.o.u.| When using awk to parse it I'm getting a bunch of ...
3
votes
5answers
230 views

How do I extract the content of quoted strings from the output of a command?

I have output from VBoxManage list vms which looks like this: "arch" {de1a1db2-86c5-43e7-a8de-a0031835f7a7} "arch2" {92d8513c-f13e-41b5-97e2-2a6b17d47b67} I need to grab the names arch and ...
4
votes
3answers
158 views

UNIX paste columns and insert zero's for all missing values

I would like to merge specific columns from two txt files containing varying number of rows, but same number of columns (as shown below): file1: xyz desc1 12 uvw desc2 55 pqr desc3 ...
0
votes
7answers
82 views

Extract the number of failures from a test report

How would I parse following output of lein test $ lein test lein test nepleaks-engine.core-test Ran 1 tests containing 1 assertions. 0 failures, 0 errors. Where I want is get number before ...
1
vote
2answers
56 views

Append a column

I've trawled through here and a few other places to try and find an answer to this, but it seems so stupidly simple that no one has answered it. All I need to do is add a single value in a new column ...
3
votes
1answer
138 views

Collapse line vertically so that every word is on a new line

Given a string like: "Hello World, how are you today?" Is there a way in Vim to collapse this line vertically such that every word appears on a new line, i.e.: "Hello World, how are ...
6
votes
3answers
165 views

Shell script to search files for identical text entries

Here's a nice teaser for a shell scripting guru: Take a directory with several text files. Can be a few up to ~1000. All files contain an identifier on a given line (always the same line). Identify ...
2
votes
3answers
137 views

How to replace the content of specific column with awk?

Given there is 40 columns in a record. I want to replace 35th column so that the 35th column will be replace with the content of 35th column and a "$" symbol. What came up my mind is like: awk ...
3
votes
5answers
122 views

Get values block by block in same file

I have a file say "SAMPLE.txt" with following content, P1 10,9:6/123456 P2 blah blah P1 10,9:5/98765 P2 blah blah P1 blah blah P2 I want a output file say "RESULT.txt" as, Value1:123456 ...
0
votes
1answer
55 views

Search a string between two timestamps starting from bottom of file

I was trying to find a string Cannot proceed: the cube has no data in huge test.txt file only between timestamps yesterday 22:30 pm to today 00:30 am. Script: tac test.txt | awk -v today=$(date ...
2
votes
1answer
49 views

comparing columnn values between two unix files

File A Table SAPSR3./1BEA/BBEA_BDH has no BasicFile LOB columns Table SAPSR3./1BEA/BBEA_BDI has no BasicFile LOB columns Table SAPSR3./1BEA/BBEA_DLI has no BasicFile LOB columns Table ...
4
votes
5answers
187 views

Replace new line + 4x space to new line in stdout

I need join 2 lines into one in stdout - replace new line AND 4x space with one space but new line without space don't touch tcpdump -vvv ... -l | xyz this is output: 2014-06-06 AAA BBB ...
1
vote
3answers
289 views

compare two columns of different files and print if it matches

I am using solaris 10 and Grep options involving -f doesnt work . I have two pipe separated files: file1: abc|123|BNY|apple| cab|234|cyx|orange| def|kumar|pki|bird| file 2 : abc|123| ...