Manipulation of text by programs, scripts, etc.

learn more… | top users | synonyms

1
vote
3answers
41 views

Selection of line from pattern, replacement with new line and incrementing number for each change

Hi I'm trying to work out how to select a line in a text file from a pattern/word string, and then replace that line with new text with an incrementing number for each line changed. I was ...
1
vote
2answers
49 views

how to trim a date to format of 00/00/0000?

I am new and have a very naive question. I want to trim and substitute the already extracted date in a .csv file from 0/0/0000 to 00/00/0000 for instance 1/9/2016 --> 01/09/2016 I have this script ...
0
votes
2answers
33 views

Remove string that has special characters

I have a file called test similar to the following: hello my name <h6>test morning</h6> is bob I know I would use: sed -i -- 's/name//g' test To remove name from the file, but how do ...
0
votes
1answer
38 views

Comparing two files and printing the content

I want to compare two files and want the output with the unique contents of both files + common content from the first file. Example :- File A.txt A=x B=y C=z File B.txt A=p D=q E=r Output :- ...
1
vote
0answers
17 views

Rejoin words breaked by hyphen and double newline characters

I have converted some PDF files to text by Calibre. Sample text as below: The booming population, coupled with a development strategy pre- mised on high levels of investment, meant that an ...
2
votes
3answers
43 views

Need to parse sections of text from repetitive “groupings” of lines within a file

I have a file that contains 3-5 lines of information pertaining to 1 instance of a Multicast group and the file contains multiple instances of Multicast groups: Group: 239.103.0.64 Source: 10.12.49....
1
vote
2answers
39 views

Joining two files matching two columns with mismatches and in each matching line, substitute second column from file 1 into 6th column in file 2

I'm struggling with this task (novice in this world...): I have two files: file1 looks like: 102 13.342 103 7.456 105 6.453 107 3.567 108 4.210 file2 looks like: 0 098 0 0 0 -9 x 0 099 0 0 0 -9 x ...
2
votes
2answers
47 views

help to construct grep output

I have a command I am using #cat test |grep "Ongoing Calls" |awk -F' ' '{print $3}' 40 2 As you can see it returns 2 values 40 & 2 I want to now add these together and put into a file as 42 ...
1
vote
1answer
37 views

Appending a line to a [section] of a config file

I've noticed that some Linux configuration files (e.g. /etc/samba/smb.conf) expect you to enter the actual settings (key value pairs) in a particular "section" of the file such as [global]. I'm ...
0
votes
0answers
63 views

Insert missing numbers in a sequence, make corresponding column default to `0` [duplicate]

I want to include any missing numbers in a sequence, and introduce 0 as default value in the corresponding columns which would otherwise be blank. Input "afile": 1 4 2 7 3 8 4 3 10 145 ...
0
votes
3answers
58 views

Want to print a table format with multiple line in shell scipt

I want to print the line in a table format using sed or awk or any other in a shell script. example Line1 Line2 Line3 Line4 Line5 Line6 Line7 Line8 Desired Output Line1 Line2 Line3 Line4 Line5 ...
5
votes
4answers
90 views

Is there an easy way to do the equivalent of `sed …` replacing the same line with multiple values?

I have a file where I want to replace some variables with data from a shell script. -A INPUT -i lo -s @LOCAL_IP@ -j ACCEPT Here I want to replace @LOCAL_IP@ with an IP address, I use the following: ...
0
votes
1answer
31 views

Command that indents lines of a C source file

I need a way to auto-indent blocks on a C source file within the terminal. According to the norms. Before: int main() { puts("Hello world"); } After: int main() { puts("Hello world"); }
1
vote
2answers
64 views

Remove lines between 2 tags (with some constraints)

Input file is like this: AAAAAAAAAAA BBBBBBBBBBB Start CCCCCCCCCCC DDDDDDDDDDD Stop EEEEEEEEEEE Stop Rec FFFFFFFFFFF GGGGGGGGGGG HHHHHHHHHHH Start IIIIIIIIIII JJJJJJJJJJJ Start KKKKKKKKKKK ...
11
votes
3answers
562 views

Merging two files, one column at a time

I have 2 large files (3000 columns, 15000 rows) of the following format file1 (tab-separated): 1/0 0/0 0/0 0/0 1/1 0/0 1/1 0/1 0/0 file2 (tab-separated): 3 5 2 1 7 10 3 4 3 I'd like to combine ...
0
votes
3answers
51 views

How to delete lines in file from specific position to the end of the file

I have a file with many lines. I want to delete from the line 1458 to the end of the file. I think that I can do it with sed, but I do not know how to.
2
votes
0answers
20 views

Copy First Column Id to different lines [duplicate]

I have a file with multiple information in different columns to the same ID first | x | y | z second | x | y third | x I want something like this: first | x first | y first | z second | x second |...
9
votes
6answers
415 views

Rearranging columns using awk

I am trying to move 7th column of my csv file to the end by using awk -F '{print $1,$2,$3,$4,$5,$6,$8,$9,$10,$11,$7}',OFS= "$file" where $file is a .csv file in a directory. However, the output is ...
3
votes
2answers
26 views

Extract parts of csv filename in columns

I have a set of .csv files in a directory, say abc_q1_w1.csv defg_q11_w2.csv hijk_q11_w3.csv I want to write a script that extracts parts of the filename and put them in columns, i.e abc q1 ...
0
votes
3answers
73 views

grep for IP without last octet as 0 [duplicate]

I am trying to extract IPs from multiple web generated lists, and need to isolate the ones that do not end with 0. Eg, in the list below, it has text, regular IPs and IP ranges. What I'm trying to do ...
0
votes
1answer
34 views

How to sum the content of all the files in directory columnwise

I want to add these files column wise i.e. column 1 of file 1 should be added to column 1 of file 2 and so on...., similarly column 2 of file 1 should be added to column 2 of file 2 ...... Col1(...
1
vote
2answers
21 views

extract part of a line before and after a string

example line: 04/07/2016 11:23:44 [8344] BDFDD123A Some Text Here (Word: ABD_ABDCD, Word: Word, Word: Word Word-ABCAbcAbc-HOSTNAME.DOMAINNAME.COM-ASDF). I'd like to extract the "HOSTNAME.DOMAINNAME....
2
votes
5answers
311 views

Reducing the complexity of the data in text file

I have this file: 1 2 2 7 3 4 4 7 5 3 6 7 7 1 8 2 9 4 And my desired output is 1 13 2 17 3 7 In my input i have 9 rows and i want to reduce it to three rows while ...
2
votes
2answers
121 views

Text manipulation: Extract everything inside brackets

I have a text file where every line is in a similar format to this: %#&#%# [Dinero] / Money / !#@%$@ [Dia] / Day / $%&$^#@ [Perro] / Dog / I am looking to extract the words ...
1
vote
2answers
77 views

Adding new columns in a csv file

I want to append two new columns in the beginning of a csv file using bash scripting in a way such that the first four letters of the filename are the values of the first column and the next two ...
1
vote
2answers
92 views

Merging contents of multiple .csv files into single .csv file

I want to write a script that merges contents of several .csv files in one .csv file, i.e appends columns of all other files to the columns of first file. I had tried doing so using a "for" loop but ...
5
votes
5answers
191 views

How do I speed up a script based on a grep search?

There is a very large text file with two comma-separated values: 78414387,10033 78769989,12668 78771319,13677 78771340,13759 80367563,16336 81634533,10025 82878571,10196 110059366,10218 110059411,...
3
votes
2answers
123 views

sed to split verilog bus into individual port

I want to convert a particular Verilog Bus into individual split form using sed or awk command. Input module test ( temp_bus[3:0], temp_B[1:0] ) input [3:0] temp_bus; output [1:0] temp_B; ...
-3
votes
1answer
45 views

replace lines from a file using script

I want to alter a few lines in a conf file.For example,the file will have something like: local all all md5 #host all all 127.0.0.1/32 ...
1
vote
1answer
41 views

extract servers' blocks from Nginx conf

I have the following conf file for nginx: user www www; ## Default: nobody ... events { worker_connections 4096; ## Default: 1024 } http { include conf/mime.types; ... server { ...
0
votes
3answers
39 views

Count the number of occurrences of specific number before each string in linux

I have this file : 90 1 120 1 Transition 150 1 Transition 165 1 Transition 180 1 225 1 240 1 255 1 270 1 Transition 285 1 Transition and I want this output: 1 2 2 1 3 1 4 5 5 ...
2
votes
2answers
156 views

Add column to a table

I have several ASCII files, having one column of data, as follows: DATA 1564 1896 1238 1479 1562 1894 1489 .... I need to implement a column containing the date. I know that the each data set ...
0
votes
1answer
39 views

How to pass string with special characters to shell command in a script?

I'm writing a small script that will help me debug some permission problems. I am passing the parent folder I wish to examine and am able to specify any sub-folders which I want to ignore. I'm ...
1
vote
0answers
24 views

convert attachments of type winmail.dat (TNEF) to “real” attachments in stored email messages, recursively in MailDir

Some email messages contain the attachments in a format called TNEF (Transport_Neutral_Encapsulation_Format). This attachment is usually called winmail.dat. Extraction tools exist for linux, but ...
14
votes
5answers
629 views

Find files that contain multiple keywords anywhere in the file

I'm looking for a way to list all files in a directory that contain the full set of keywords I'm seeking, anywhere in the file. So, the keywords need not to appear on the same line. One way to do ...
3
votes
2answers
158 views

`cut`: selecting columns containing a string

I have a big file with several columns on each line. I'm familiar with using cut -f -d to select specific columns by their number. I checked the manual for cut and it doesn't seem that there's a way ...
1
vote
2answers
33 views

using gsub in awk to replace a word with parentheses

Hi how can I use gsub to replace a word which has parentheses. Here I want to replace ABC(T) with ABC/G awk ' {gsub("ABC\(T\)","ABC/G")}; Print $0' "$FILENAME" > tmp.tmp && mv tmp.tmp "$...
4
votes
4answers
354 views

How to print lines between pattern1 and 2nd match of pattern2?

Test file is given below: PATTERN1 a b c PATTERN2 d e f PATTERN2 g h I want to print line between PATTERN1 and 2nd match of PATTERN2: PATTERN1 a b c PATTERN2 d e f PATTERN2
3
votes
1answer
77 views

Split file by pattern keeping only the pieces that contain a second pattern

I found the following awk script in a previous answer for splitting large files into smaller parts based on a pattern (replacing "header-line" with a different pattern). However, I have the need to ...
1
vote
3answers
94 views

Combine matching columns from two files, or use default value

I am trying to rewrite a file text, main.txt with changes from changes.txt. The main.txt file is like this: word_1 genre_A word_2 genre_A word_3 genre_B word_4 genre_C word_5 genre_A And the ...
1
vote
1answer
49 views

Better way to apply formula to specific column of inputfile, and pass-through all other columns as-is

I regularly need to apply simple formulas to input data give in textual tabular form. In the beginning, I wrote specific little functions for each use-case. Then I re-factored the "common code" into ...
6
votes
1answer
144 views

ANSI code translator

Is there any tool that translates text with ANSI code to text-like output? tput cap names or the like? Reason for the inquiry is that I would like to use script or the like and view the resulting ...
1
vote
3answers
53 views

Substitute a column i text file with incrementing numbers

I need to do something very similar to this Replace string with sequential index, but I don't need to add a number to a column, but substitute a whole column with incrementing numbers. Like this: 0 ...
0
votes
2answers
40 views

How to extract string between Nth and Mth occurrences of a pattern?

I have a file path.txt with the path to a file(say a txt file): /Folder/F1/F2/F3/F4/.../Fn/abc.txt I want to extract the path i.e the string from beginning to the last occurrence of / (since the ...
4
votes
2answers
61 views

Compose string of IPs using bash command line

I have a command that returns the following: id1="172" id2="17" id3="136" id4="5" id5="0" /> id1="172" id2="17" id3="128" id4="2" id5="0" /> id1="172" id2="17" id3="128" id4="4" id5="0" />...
4
votes
5answers
307 views

Add space between files paths

I have a list of folders and inside every folder I have the file " file.txt" which has the same name in all the folders. I wanted to collect the path for this file from all the folders and save it in ...
3
votes
2answers
123 views

Add a line below each line matching a pattern only if not already present

Can sed add a new line below a specific content, if the input content exist then leave it ? Current content of the file ssss Hostname example.com Os version rhel5.6 apache 4.2 Hostname example2.com ...
1
vote
1answer
49 views

Extracting from Logs

Actual logs are: 2016-06-19 22:08:09 [213917] 1bEgCe-000tZR-E9 ** saadia@aol.com (fahad@manakintextiles.com) <fahad@manakintextiles.com> F=<james@designplus.com> P=<james@designplus....
0
votes
2answers
35 views

BASH get value from string

I have this string stored in a variable <li class="rainbow-list-item southern disrupted " data-line-id="lul-southern" data-severity-codes="4,6,9" data-mode="national-rail"> and I would like ...
8
votes
5answers
673 views

How to find lines that begin with **

I need to find if any lines in a file begin with ** . I cannot figure out how to do it because * is interpreted as a wildcard by the shell. grep -i "^2" test.out works if the line begins with a 2 ...