1
vote
2answers
53 views

Editing a Java properties file with Bash

I have the following Java properties file that is maintained by humans. # file.input = /very/old/name # file.input = /old/name file.input = /current/name file.input.default = /default/name ...
1
vote
2answers
43 views

Bash: split list of files into different files?

I have several files in a directory like: data1.dat data2.dat .... data24678.dat To run another code, I need to have a text file with the names of the files but I need to split in 50 files per ...
3
votes
1answer
40 views

Find and Replace Recursive

I want to modify every file in a directory recursively. I want to replace: <?php with <?php /** * * Copyright (C) MyCompany, Ltd. - All Rights Reserved * Unauthorized copying of ...
2
votes
3answers
56 views

How to optimize this oneliner? (if second column empty, put the line in another file)

I have a little "oneliner": while read ONELINE; do if [[ -n "`echo $ONELINE | awk '{print $2}'`" ]]; then echo "$ONELINE" >> LIST-ok.txt else echo "$ONELINE" >> ...
5
votes
1answer
94 views

counting multiple patterns in a single pass with grep?

I've written a grep loop to iteratively count DNA trinucleotides within a gzipped DNA fasta file containing DNA sequences e.g. declare -a tri=(AAA AAC AAG AAT CAA .. etc) for i in ${tri[@]} do ...
3
votes
4answers
75 views

Replace all newlines to space except the last

How can I replace all newlines with space except the last newline. I can replace all newline to space using tr but how I can do it with some exceptions?
1
vote
1answer
229 views

grep to extract a substring from a huge string

I am extracting certain information from a website using curl command. I need to extract a certain subsection of the string using grep command. For example, using curl command I am getting the HTML ...
2
votes
1answer
30 views

Count and merge consecutive patterns

I'm searching for a short snippet to find, count and merge consecutive duplicates using standard tools or a common scripting language. Say our input is: 1 2 2 2 7 22 a b b c c c c c d dd 2 2 c c ...
0
votes
2answers
60 views

Using AWK to delete line and output result

I have a text file and manage to find the line through output comparison. With the book title and author name. awk -F ':' '/KungFu Feet|Chuck Norris/' test.txt Kungfu Feet:Chuck Norris:12:1:1 Is ...
3
votes
5answers
88 views

Join two files, matching on a column, with repetitions

How can I get two files A and B, and out put a result like this: File A: 001 Apple, CA 020 Banana, CN 023 Apple, LA 045 Orange, TT 101 Orange, OS 200 Kiwi, AA File B: 01-Dec-2013 01.664 001 ...
1
vote
1answer
58 views

removing data from txt file

I have a comma-delimited giant file that was exported from a database. The file has this format Date,n1,n2,n3,n4,n5,n6,color,type,reference 10-Jan-2000,344,13,21,2,55,3,"rose garden" ,3444,32 ...
2
votes
3answers
42 views

To assign variable from a file separated by delimiter

I have a text file with datas as below separated by delimiter: 1000:Tom:user1 1002:Jerry:user2 1003:Victor:user3 1004:Jason:user4 I need to run a while loop (any loop) to fetch details from first ...
5
votes
1answer
94 views

How to append multiple lines to a file without last newline?

I have a very long string that is split in chunks. I want to append them to a file without putting the newline character using bash. Example: First append cat >> abc.sh << EOL echo "bla ...
2
votes
2answers
80 views

Show a sliding window of output from a program

I want to run a command (rsync -av) and only display 10 lines of output at any time. I have found a similar post, but I don't want to redirect it to a file. I want the output to be displayed ...
1
vote
2answers
55 views

Mailing out a customised content based on specific search in a logfile

Using below test.sh, I am getting result in dump.txt as below: test.sh: #!/bin/bash #"Maxl-script" zone starts..... essmsh -l admin password -s localhost -i << EOF spool on to 'dump.txt'; ...
0
votes
2answers
72 views

How to Lookup the values of an index values from a big data file?

I have a data file (data_array.txt in the example) and index file, index.txt, at which I want to extract the data from the data file that have the same ID in the index file and store in to a new file, ...
1
vote
5answers
250 views

Cut and transpose some lines of a file

I have 1344 rows of data column. I want to form 32 row by 42 columns from my input data file. I want to cut the first 42 row and paste to one raw of 42 column data and continue down likewise. My data ...
9
votes
4answers
99 views

Script to extract selected entries from a bibtex file

I have a large bibtex file with many entries where each entry has the general structure @ARTICLE{AuthorYear, item = {...}, item = {...}, item = {...}, etc } (in some cases ARTICLE might be a ...
4
votes
3answers
119 views

Parsing string token with bash

I have this output (from a previous command): aaa something bbb someother ccc blabla For each line, I would like to create a file whose name is the first token, and whose ...
0
votes
1answer
102 views

Copy file while replacing text in it

I'm writing a script to publish a webapp. While copying files, I must replace a placeholder current_date in a file with the current date. I would start with something like this to define the date ...
4
votes
2answers
88 views

Remove duplicates from basenames in two files

I have two long lists of filenames, and I want to merge these two by excluding duplicated filenames in the sense of basename. For example, I have something like ../data_folderA/file_1 ...
-1
votes
2answers
129 views

How to manipulate output from commands with shell scripts [closed]

Simple question, having a ls -l command it makes a list on the command prompt and if we want to manipulate it we pipeline it with awk command, for instance if we want to to show specific files. Is ...
4
votes
4answers
278 views

how do I trim each line in bash?

I would like to remove all leading and trailing spaces and tabs from each line in an output. Is there a simple tool like trim I could pipe my output into?
5
votes
3answers
199 views

removing redundancy from output columns

What is a *NIX way of removing redundancy in a case where I have pairwise comparisons like these in two columns A B B A A C A D C A D A B C C B A B and B A ...
1
vote
3answers
99 views

Extracting mail addresses

I have a file that contains lines like this: user=<xxxxxxx@asd.com.tr>, user=<yyyyyyy@asd.com.tr>, user=<zzzzzzz@asd.com.tr>, user=<ccccccc@asd.com.tr>, ...
2
votes
5answers
105 views

Read contents in a line in bash

I have a file with contents in this order which is separated by space 4096 1230 4560 1024.... n How can i read each number and pass it on to another command, eg divide each number by 1024.
3
votes
1answer
113 views

Simple Bash Script

I have a simple problem, but unfortunately I don't know where to even start (I'm just starting out). So, what I want to do is ultimately increase my vocabulary. I got the idea to strip the most ...
1
vote
3answers
198 views

compare files line by line and create new one bash programming

I have two text files. File 2 has logs over 1,000,000. File 1 has IP addresses line by line. I want to read file 2 lines and search these lines in file 1, I mean: file 1: 34.123.21.32 45.231.43.21 ...
3
votes
2answers
177 views

How to find frequency of occurrences of strings contained in a file?

I have a file that contains a list of URLs of the form EDIT http://www.google.com/absd/siidfs/kfd837382$%^$&, www.google.com, google.com yahoo.com/list/page/jhfjkshdjf... I ...
4
votes
3answers
374 views

using sed to replace pattern with hash values

I want to search the file and replace specific pattern with its hash (SHA1) values. For example, let file.txt has the following content: one S56G one two three four five V67X six and I want to ...
1
vote
1answer
561 views

C shell script : how do I compare two strings?

I'm a C shell user. I've two files. file 1: A B C D E file 2: A C B D E I want to compare only A-A, B-C, C-B, D-D, E-E using a for loop. If the strings match, then print "match". The output ...
3
votes
2answers
158 views

Bash-script. Shift seconds

Maybe somebody can help me. In bash I don't know how to do that. I need to do a bash-script. At stdin I have .srt file of subtitles in this format: num HH:MM:SS,SSS --> HH:MM:SS,SSS text line 1 ...
1
vote
1answer
110 views

Binary search for a failing line

git has bisect run to figure out which revision an error was introduced. I have a big file (100GB) and at least one of the lines is bad, but the program I have to check it with will not tell me which ...
3
votes
4answers
504 views

Align text to center with padding on both sides

Is there an easy way to align text to the center with padding on both sides, with the column width being the longest line from the input? For example, this: aaa bbbb c ddddd ee will turn into this ...
1
vote
2answers
292 views

Printing fields using awk

< pool.sam awk ' /./ {printf $1} {printf $7+1,"\t"} {printf $3,"\t"} { if($2 !=16) {print "\t", "+";} else {print "\t","-";} {printf $4,"\t" ,length($10)+$4, "\t", "1"} }' I am ...
1
vote
5answers
288 views

Print nth line before the matched line, Matching line and nth line from the matched line

I want Print nth line before the matched line, Matching line and nth line from the matched line where "n" is greater than 2. Here's an example of my data file (the line numbers below are not part of ...
1
vote
1answer
98 views

Parsing the headers of sequence file

I have a multiple sequence file as >abc|d017961 sequence1...... >cdf|rhtdm9 sequence2...... >ijm|smthr12 sequence3...... >abc|d011wejr sequence4...... >stg|eethwe77 sequence5...... ...
1
vote
2answers
77 views

Split out data from a column

So at the moment I have some input email addresses and would like to separate the alias and the domain, then copy them to the end of the line. For example, from a line that has ...
2
votes
3answers
549 views

Comparing files line by line in awk with delimiter

file1: abc|123|check def|456|map ijk|789|globe lmn|101112|equator file2: abc|123|check def|456|map ijk|789|equator lmn|101112|globe EXPECTED OUTPUT: ijk|789|equator lmn|101112|globe Current ...
1
vote
2answers
239 views

Improve sed command to replace first instance of character and all following characters?

Input: <e1 name="file1" id="id1" anotherId="id2"> Desired output: file1 I can get what I need with this: echo '<e1 name="file1" id="id1" anotherId="id2">' | sed ...
6
votes
4answers
2k views

Compare two files with first column and remove duplicate row from 2nd file in shell script

I will ask my question with an example. I have 2 files: File #1: 118D FC300_R5_TP FX.B 32775 2112 6 2038 6 2112 0 118E FC300_R5_TP FX.B 32775 2136 7 ...
4
votes
4answers
225 views

Joining two files with unique identifier

I have two files with approximately 12900 and 4400 entries respectively, that I want to join. The files contain location information for all landbased weather observing stations around the globe. The ...
2
votes
2answers
79 views

selecting specific string in a field

I have this file: N104 F1810 N104 F3810 N104 F810 N104 F305 and a script to grep an input. Script: enter input read input cat file | grep $input when I input “810”, I get the ...
2
votes
2answers
3k views

Loop over a file and read values from two columns into variables

I have a file (data.dat) which contains 3 columns. I want to read in two of them with the shell script and fill the contents into a template file. The file looks like this: 12 180 2390 14 177 2210 16 ...
3
votes
5answers
167 views

Comparing Indices of two strings

I have two double quoted strings of same length, that look like this: "$con" - (consists of spaces and *'s): * ****** *** ** * ** and ...
2
votes
3answers
234 views

How to grep the last number on a line in a bash script

I have output from the smartctl -a command; one of the lines looks like this: 9 Power_On_Hours 0x0032 100 100 000 Old_age Always - 301 I've placed this into a variable (if that's the correct term) ...
0
votes
1answer
90 views

Removing lines based on tags for different files and merging into single file

I have to develope unix script in such way it should meet below scenario. I have 3 text files, 1.txt , 2.txt and 3.txt Data in those files as below. HDR|1|2|3||||||||||||||||||||||| ...
1
vote
2answers
82 views

Remove complete line in text files based on Tags

For example : 01.txt, 02.txt, 03.txt and 04.txt has following values. HDR|A|B|C|D|F||||||| DTL|1|2|3|4|5||||||||||| DTL|5|6|7|8|9||||||||||| TLR|A|B|C|D|F||||||| I have to remove complete lines ...
1
vote
1answer
47 views

Pipeline each line of a file to a specific variable in a second command

I want to execute the following instruction: basex -bword=ENTRY consulta.xq But for every line of a SampleText file (plain text mainly). The SampleText contains the following lines: hello evening ...
4
votes
2answers
965 views

How to remove lines included in one file from another file in bash?

I have two lists, a complete list and a partial list. I want a bash script that will search the complete list for any lines that are in the partial, if there are, they will be removed from the ...