Manipulation of text by programs, scripts, etc.

learn more… | top users | synonyms

2
votes
6answers
128 views

Comparing 3 lines and 2 lines in BASH

I have a text file which looks like this: 2014-11-24 12:59:42.169 101.0.0.0 source 2014-11-24 12:59:40.375 104.156.80.0 destination 2014-11-24 12:59:36.729 104.219.48.0 destination 2014-11-24 12:59:...
2
votes
3answers
247 views

Append data of multiple files to new file separated by a line

I have more than a hundred firewall rules for many different customers in discrete files in a folder. Suppose the filenames are: 1.wawa.com_firewall_rule 2.newa.com_firewall_rule Each file ...
1
vote
2answers
64 views

Merging Two Files Columns

I want to add 6 field of File2 in last field of File1, i'm using following command but unable to get output. awk 'FNR==NR{a[$1]=$6; next} {print $0,a[$1]}' File2 File1 File1: zehriscollection.co....
7
votes
3answers
672 views

merge two files, first line from first file followed by first line from second file [duplicate]

I have two files : f1: f2: ============== =============== some text line 1 A1 some text line 2 ...
2
votes
1answer
170 views

How to filter lines passed through a pipe?

I've got a command foo that outputs a list of filenames: $ foo file1.a file2.b file3.a And a command bar that accepts the names of .a files as arguments and does some processing: $ bar file1.a ...
3
votes
1answer
54 views

List names of files containing a string that occurs between two other strings

I need to print a list of files with match a specific pattern -- a string occurring between two other strings. How do I do that. I don't want to extract text, just list the files that match this ...
2
votes
1answer
50 views

How to choose the highest number element in many files with many arrays

I have many files which each one contain many arrays(like input 1 and 2, below). Each array has two element numbers in the second row. I need to use a logical criterion to choose the highest number ...
3
votes
1answer
67 views

awk or sed command to start each column of file from desired locations or Byte Position

I have a source file format like below containing 15 columns and so many rows: 23.5 31.5 5.00 255 0 255 2 1 sal unknown sa time sps meter ms 25.5 32.5 6.00 0 255 0 2 2 sal unknown sa time sps ...
-1
votes
1answer
55 views

Extract Information From Logs

Unable to grep third field "Email From": While I'm using this command to grep first two fields: echo "TimeStamp Email To: Email From:" && awk '{print $1,$6}' logs Logs: 2016-05-23 11:01:...
-1
votes
1answer
39 views

Extract string from first character till string pattern

I have a implicit file with **$&*#@** as field delimiter and **$#@&*** as record delimiter. Below is an example, 100$&*#@200$&*#@50$&*#@Some String$&*#@2016-01-01 101010$#@&...
1
vote
1answer
42 views

Replace only the first character of a matched pattern

I need to write a sed command that reads from a file and replaces the first character in each line with 0, only in the event that the first and second character in that line is a digit (and leave the ...
2
votes
2answers
50 views

How to check if input contains all 26 chars

I got a input file which contains one char per line I want to check if the file has exactly all 26 letters. I found that uniq -d can help to get the repeated one. But how to find out the missing ...
2
votes
1answer
21 views

Strings from two distinct arrays not matching?

I'm trying to filter one array from another array. That is, I'm trying to create a third array with a logical not-intersection. The best I can tell, it appears this block of code never matches, and ...
1
vote
0answers
22 views

Compare Two File And Match Column One [duplicate]

Compare two files if second file contains a domain which is not found in first file than add "Static" in second field. File1: cleanwatersaveslives.com WordPress blood-mate.com ...
2
votes
3answers
132 views

Extract a line if the first field contains a dot

The actual data is: Dolibarr techpubl http://techpublications.org/erp tekstilworks.com WordPress tekstilw wbq.dandydesigns.co WordPress ...
0
votes
1answer
44 views

How to multiply numbers located in two different files

I have two files ... First file(input 1): 1: 6.1703 44 14.7262 46 18.3255 2: 6.1932 44 52.9379 46 4.30653 3: 6.5664 45 2.82839 46 3.1275 4: 8.0923 44 14.1076 Second file(input 2): 43 ...
1
vote
2answers
22 views

How do I swap two fields separated by backslash and output them separated by an at sign?

Original file contains email addresses in the following format: domain.com\first.lastname I have used sed to replace the backslash with an @ sign. I've tried various suggestions to swap the domain....
0
votes
0answers
31 views

add a column with data to a file based on the same name in another column [duplicate]

I would like to add an additional column(s) with some info to a file based on the same names in the first column: First file: OTUId 1 2 3 OTU_93 -2413 -2209 -878 OTU_19 -1426 -1388 -...
2
votes
2answers
124 views

How to multiply the array elements by itself

I have an array file and I need to do a math operation with the elements. The operation that I need to do is to multiply the element by itself, the to print it. The input looks like this: 1: 6.1703 ...
1
vote
2answers
63 views

Convert command output to csv, with time stamp

I want to test a speed connection with terminal command: ./speedtest-cli And it returns This: Retrieving speedtest.net configuration... Retrieving speedtest.net server list... Testing from Moscow,...
0
votes
3answers
63 views

Regex pattern to replace multiple values via sed

I have a script in which I need to parse the IP address of the localhost, and replace the 2nd octet with a different value, but the value being substituted depends on what the value currently is... ...
3
votes
3answers
53 views

Grep/awk/sed for lines composed of only two letters, and lines that start with a letter and meet a certain length

Not sure how many people are familiar with DNA sequencing data, but if this is part of my file (lines starting with ">" are IDs, and lines starting with a letter are DNA sequences): >NB501013:9:...
1
vote
4answers
70 views

Extract X number of minutes from the current time from a logfile

I have a logfile with the following timestamp format 2016-06-02 13:59:58.069 The log I am working with spans about an hour. 2016-06-02 13:05:06.144 2016-06-02 13:05:06.144 2016-06-02 13:05:06.160 ...
2
votes
2answers
47 views

Print text between two strings - omit second occurrence

I have an instruction file with a lot of steps. I'd want to view a single desired step in a shell, without graphical environment. What I do right now, as pointed out by alemol, is using sed to extract ...
1
vote
1answer
49 views

How to sum specific rows of a matrix

I need to sum the matrix elements of specific rows for each column. An example of the matrix is given below(input). So the matrix has 4 columns and 6 rows of numbers. I need to sum the numbers of ...
1
vote
4answers
104 views

How can I cut the character from “Thu Jun 2 08:11:53 PDT 2016” such that to display only 08 11

How can I cut the character from "Thu Jun 2 08:11:53 PDT 2016" such that to display only "08 11"
0
votes
2answers
64 views

I want to edit the text of a specific file

I would like to modify the configuration file, as shown below. What can I do as a shell script ? before Section "InputClass" Identifier "calibration" MatchProduct "Touch" ...
0
votes
2answers
62 views

Comparing two lines in BASH using awk

I have text file like this. 2015-11-24 12:59:37.112 128.206.6.136 source 2014-11-24 12:59:36.920 8.8.8.8 source 2014-11-24 14:59:38.112 23.234.22.106 destination 2014-11-24 13:59:37.113 23.234.22.106 ...
1
vote
4answers
66 views

How to use Unindented line as the Record Separate in awk cli

I have a log file that look like: 2016-05-31 09:54:36 (16667) heritage_w? From: ip68-8-49-100.sd.sd.cox.net User: user1wizard (wizard) Agent: Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/...
1
vote
1answer
81 views

Summarize certain data fields of apache CLF log file

I want to grep some information from raw combined log format apache logs: 51.254.56.62 - - [01/Jun/2016:20:49:28 +0500] "GET /vendors/jquery.slimscroll.min.js HTTP/1.1" 404 - "http://networkconfig....
3
votes
2answers
71 views

Compare lines and upgrade two different files

I have a couple of text files with the following features: $ cat file_1 Line A Line B Line C Line D $ cat file_2 Line A Line added 1 Line B Line D Line added 2 They are such that file_1 has some ...
0
votes
2answers
32 views

Interchange two columns

I am trying to swap the "Hits" and "Domain" column. cat filename Output: Hits Domains 204492 greenlivingasc.in 37102 smcracks.uk 17410 bahriains.org 10975 ...
1
vote
2answers
44 views

How to append some string combined with the pattern at the end of line with the pattern

I want to append .com at the end of line for every line matching the pattern in /etc/hosts file. Sample file content: 127.0.0.1 localhost 1.2.3.4 hostname1 hostname1.xyz hostname1.xyz. 1.2.3....
1
vote
2answers
47 views

Compare string in two files And Merge its Output

I am trying to get the bandwidth of the domain from one file and its hits details from another file. The format of the two files is as follows: bandwidth.txt aadrivingschool.ws 2840.36M ...
1
vote
0answers
37 views

How tcan I delete columns from a file without losing the format? [closed]

I have a file with this information: 1 2 3 4 5 O O O O O Eigenvalues -- -20.44628 -...
0
votes
1answer
27 views

How to move lines between patterns to the end of a file

I want to move several lines to the end of the file. Example: 1 2 3 4 5 6 blkn XYZ indf 7 8 9 10 11 lnkn ABCYDF knnfd 12 13 14 15 I want to move lines 6 to 11 to the end of the file. The patterns ...
3
votes
6answers
1k views

How do I remove a words ending in the letter S if duplicates without an S on the end are on the same list?

I have a large list of words. Many of the words are only different because they have the letter s on the end. If a word on the list is the exact same as another word on the list, except one of the ...
2
votes
1answer
73 views

How to sum numbers inside many 2D arrays

I need to sum numbers located in a file like this one: column1 column2 column3 row1 a(1,1) a(1,2) a(1,3) row2 a(2,1) a(2,2) a(2,3) row3 a(3,1) a(3,2) a(3,3) row4 a(4,1) a(4,2) ...
1
vote
3answers
40 views

How to delete whitespace surrounding equal signs in a string?

I'm trying to determine a way to delete leading and trailing whitespace surrounding the equal sign from a string. The whitespace may (or may not) be present. There may be 0, 1 or more equal signs. I'...
3
votes
4answers
114 views

All possible permutations of words in different files in pairs

I have multiple files, let's say file1, file2 etc. Each file has one word in each line, like: file1 file2 file3 one four six two five three What I want is to combine them in a new file4 in ...
-1
votes
1answer
24 views

Replace values with #<value># with date - '01-Jan-2015'

I want to replace values in file. Replace : values #startdate# with - '01-Jan-2015' values #code# with - 'SA','CA'
1
vote
1answer
45 views

Need to filter the data on the basis of two column having many to one relationship using awk

I have a big file having 50s columns and 100K of rows delimited by |. Now $2(col 2) has multiple type of $1(col 1) value which means col 2 will be repeated. So I have sorted the file. I need now to ...
-1
votes
2answers
64 views

how to change record or field separator of a file [closed]

How can I change the record separator of a file? For example, concatenating multiple lines into a single line, by changing record separator from newline to space. Another example is to split a line at ...
1
vote
3answers
58 views

Remove all the rows below a specific row number from one specific column

I've asked a question before on true row deletion, and got some great answers, but I'm now interested to know how you would remove all the rows below a specific row number from one specific column. ...
0
votes
1answer
24 views

extract two columns and add at a specific location to another file that is already full of columns

I have a file where I want to extract two columns and add at a specific location to another file that is already full of columns. Could I cat file 1 and make it a variable and then cat file 2 and ...
6
votes
4answers
75 views

True column deletion, not selecting all columns that you want

I know this is a simple question and in many ways has been answered before, however every use I've seen of awk, sed, grep etc have all been selecting and printing the columns you want. In a case where ...
2
votes
2answers
53 views

add a file name to lines in this file

I have fasta files, coming from different samples eg: 01.fasta, 02.fasta, etc. All of them contain OTU list with the following pattern: >OTU1 CCTACGGGCGGCTGCAGT >OTU2 CCTACGGGTGGCTGCAG I ...
2
votes
2answers
47 views

I would like to replace a pattern with another pattern found in the same line

I would like to replace the second tab-delimited field with the first underscore delimited field found in the same line Input Cho1_01:C4WMNACXX:1:250326554 1 2 4 2 -9 1 1 4 3 1 4 1 -9 4 1 4 1 3 ...
0
votes
1answer
29 views

How to insert fields from one file into fields of another file at particular location?

Is there any quick way to inserts fields from one file into another. For example, if the file contents of file1 and file 2 are like this : File1: ======= size1 size2 size3 File2: ======= char ...
1
vote
4answers
40 views

cut column 2 from text file

My text file has no delimiter to specify separator just spaces, how do I cut out column 2 to output file, 39 207 City and County of San Francisc REJECTED MAT = 0 78 412 Cases and ...