Manipulation of text by programs, scripts, etc.

learn more… | top users | synonyms

5
votes
6answers
517 views

split file into two parts, at a pattern

How to split a large file into two parts, at a pattern? Given an example file.txt: ABC EFG XYZ HIJ KNL I want to split this file at XYZ such that file1 contains lines up-to XYZ and rest of the ...
0
votes
3answers
81 views

Reorder lines in each 3-line record

I have a file which contains lots of records in following manner. Name Address Phone This is the sequence for each record in that file. Now, I want to copy this file to other file with changed ...
2
votes
2answers
119 views

Extract substrings from each line. These substrings are in comma separated, “name=value” format

I have lines in a file that are comma separated. There are no column headers, it's mostly comma separated 'name=value' pairs. Here is some test data: ...
2
votes
3answers
70 views

Reformat date string

I am trying to figure out how to reformat a file full of wrongly formatted dates. The source looks like this: {"_id":"","timestamp":"Mon Apr 20 08:30:55 +0000 2015"} {"_id":"","timestamp":"Mon Apr 20 ...
0
votes
1answer
65 views

How can I regex specific links from rss content in yahoo pipes?

From a RSS content I'd like to remove everything except specified href links Example: <div class='text'> <div class="center"> <b><a rel="nofollow" target="_blank" ...
-1
votes
2answers
134 views

How to split output lines into multiple lines (only lines not files)? [duplicate]

My requirement is display output lines into mutliple lines My code is: #!/bin/bash dir="$1" echo -n "file size:" du="$(du $dir -hab | sort -n -r | tail -1)" printf "%s\n" "echo "$du"" It's ...
1
vote
2answers
138 views

How to check each line within a pipe delimited file to be same of different

I have a pipe-delimited file which can have millions of records. What I want to do is sort the file and remove if there are any duplicate lines there. Then I need to check the first 32 columns and ...
1
vote
2answers
89 views

Search and replace multiple strings in a script using vim(search and replace) commands without sed?

A script that creates a file for weather precipitation is, #!/bin/ksh x=10 b=snow c=rain d=snow e=hail echo "$x,$b,$c,$d,$e" > weatherfile exit 0 given 3,8s/snow/dry/g For the search and ...
1
vote
2answers
40 views

Copy line in file if we know the pattern [closed]

We have a file which has 10 lines, and I know that some line has text QWERTY. How can I manipulate the file so that it copies that line and paste it. The output is similar to that yy and p command ...
0
votes
1answer
59 views

Help in manipulating a test file

I have a file look like aaa/bbb/ccc.bbb aaa/bbb/ddd.bbb aaa/eee/fff.eee aaa/eee/ggg.eee I would like to skip lines ending similarly (where ending is defined by the extension, e.g. .bbb & .bbb) ...
0
votes
3answers
77 views

Removing line-breaks, whitespaces and tabs from every file

Say I have a folder with 10K text files. I would like to remove every space, TAB and linebreak from each file. How can I do this efficiently?
1
vote
2answers
86 views

AWK script to compare content of 2 files

I need experts help in resolving my following problem ... I have 2 files .. file1: abc|123|check def|456|map ijk|789|globe lmn|101112|equator file2: check map equator globe AWK function ...
2
votes
3answers
243 views

add space 5 times at the beginning of each line in a text file

I am looking for a command line or bash script that would add space 5 times before the beginning of each line in a file. for example: abc after adding spaces 5 times abc
0
votes
2answers
64 views

Using sed, add line X lines above match

I know that to prepend/append something, I just have to do: sed -i '/pattern/a new thing!' However, I'd like to match a line, and prepend something two lines higher. I'd like to limit tools only ...
0
votes
2answers
50 views

Order the output using sed [duplicate]

I need your help using sed. I have this output after executing diff: < #R1#Number = Gauge32: 258 Name = STRING: "TATA" --- > #R1#Number = Gauge32: 280 Name = STRING: "TATA" I need to ...
0
votes
3answers
110 views

Loop for snmpwalk in bash

I have an issue in my output using snmpwalk so i'm trying in this script: var=`snmpwalk -v2c -c private ${ip} .1.1.1.6` echo "${hostname}=>${var}" to have this output: R1=>Gauge32: 240 ...
0
votes
3answers
81 views

Remove lines in a shell script

I have this file: # more file.txt 2c2 lns-ld-wall-01-t2 old:261,260 4c4 Prive_ORANGE old:258,259 I need to remove all lines like this one 2c2 and this one 4c4. How can I do in a shell script?
6
votes
3answers
130 views

How to use sed to manipulate continuously streaming output?

I am putting together a presentation for a non-technical audience. I have a program running in bash that outputs a continuous stream of values, a few of which are important. I would like to highlight ...
0
votes
1answer
30 views

How can I match and substitute this multiline pattern in a bash script?

I have a file jail.local and it has sections like this [apache] enabled = true logpath = /var/log/apache/error.log [ssh] enabled = false port = ssh I want to match on the string [ssh] and ...
1
vote
2answers
62 views

Using Awk to Search Bro Log for IPs/Keywords Found in Online Text File

I am examining Bro logs and would like to find a way to use AWK to grab IPs from a tracker link and search the Bro logs for those IPs automatically, printing the results of each match into ...
3
votes
3answers
145 views

Shell script to move files if certain percentage of lines contain a certain string

I am working on a shell script to move files if certain percentage of the lines contain a certain string. I've got ~2000 files in a directory that each contain a single column of data. The number of ...
3
votes
1answer
129 views

Command for killing specific PID provided by previous command

Sometimes I need to kill a process (the reasons why are not important). And I know I can find that process with the following command: lsof -i :8080, being my candidate the last process in the output ...
2
votes
1answer
101 views

Filter a log file by month, date and time in bash

I'd like to fetch the data between Apr 24 10:00:00.000000 and Apr 25 24:00:00.999999 in the following log but I am not sure how to get this work: files/file1:Apr 22 02:47:00.663117 somedata ...
1
vote
1answer
39 views

How to filter a basic csv by columns

How do I filter by third column of the following file list.txt [[ list of people ]] billy baxter - @baxter - b.baxter@woods.com james woods - @woods - j.woods@someother.com I currently have cat ...
1
vote
1answer
92 views

Awk Command vs Grep

I had a question regarding the awk command. I want to take a bunch of files and find a single line from each and extract them into a comma seperated text file such that I can import it into excel for ...
1
vote
1answer
65 views

Read data from a file in a csh script

Is there a way to read data from a csv file into a Cshell script? I have a small script, which calls another script recursively with a set of parameters. I want to read this set of parameters from a ...
2
votes
3answers
114 views

Using sed to consolidate diff output

I have this output as a result of comparing files with diff: < IF-Name :STRING: "lns-wall-01-t2" Index:Gge32: 260 --- > IF-Name :STRING: "lns-wall-01-t2" Index:Gge32: 25 I need ...
6
votes
4answers
193 views

Extracting subset of lines of a file based on regex for first & last line

I have a large text file and I only want to look at some of the lines. The first line I want matches a regex R, and when the line matches the regex S, I don't care about that line, or any following ...
2
votes
2answers
196 views

Extract group name from `id` and store it in a variable

$ id usera uid=830(usera) gid=799(groupa) groups=799(groupa) I need to extract the group name from the output of id and store it in a variable. In this case it's groupname=groupa
1
vote
1answer
45 views

Merging rows with common values

I am trying to combine pairwise matches in a text file so that i can find all matches in a group: e.g. my file contains two tab separated columns like this: Simon John Simon Paul Steve Simon Graham ...
4
votes
2answers
128 views

How to find out which process keeps tunnel interface(tun) up?

If I create a tun interface with ip tuntap add mode tun command and force it administratively up with ip link set dev tun1 up command, then the interface itself is always "physically" down: ...
1
vote
3answers
118 views

How to copy lines from multiple files into one new file and keep file name?

I have 81 files in .fasta format that contain (up to) 53 items. Such as: /User/MyData/Sample_1.fasta /User/MyData/Sample_2.fasta .... /User/MyData/Sample_81.fasta Each .fasta file contains a name ...
1
vote
2answers
49 views

Reinterpret terminal output

I have a program that executes some commands in a terminal (for example apt-get update) and stores the output. How can I reinterpret the output, so I can remove all information that is not needed ? ...
0
votes
1answer
96 views

need help merging two Unix files

I have 2 pipe delimited files say file1 and file2. Where file1 has say 33 columns and file2 may have 34/35/36 columns. So let's not consider how many columns we have. What I want to do is compare the ...
0
votes
1answer
89 views

make a table output

if it possible to do a snmpwalk in a perl script and put the output in a table to make a sort of association like for each hostname i have in the same line if index and desc i have the script in bash ...
4
votes
3answers
177 views

Redirect the tail output to a file then stop when match found

there is this log file that data coming in is continuous. what i wanted to happen is to tail -f this log file then redirect it to a file with the ff conditions example of the log files content ...
0
votes
1answer
106 views

Run script on multiple hosts [duplicate]

I have below script which creates user list and also filters ignore.txt file to ignore matching usernames. I would like to run this script on multiple hosts like hostA, hostB, hostC. Like if I run on ...
0
votes
2answers
72 views

put data in columns

I am trying to put this output in columns can anyone help? Routeur 2 ==> 89.89.156.112 If_Name :vRtrIfName.1.1 :vRtrIfName.1.2 Routeur 2 ==> 89.89.156.112 If_index 546 789 Routeur 2 ==> ...
0
votes
1answer
49 views

Add the number in the column if exists

I need to count Nodes_Util for each Queue. In the example below, queue1 has only one line, but it varies and it could be nothing sometimes and it applies to all queues. I want to dynamically add the ...
1
vote
6answers
89 views

Count the number of 0 and 1 in a file [duplicate]

I have a file (see below), and I would like to count the number of 1's and of 0's: 1 1 1 0 0 1 1 0 0 0 ... I tried to use awk but it doesn't work, for example, to count the number of 1's: awk -F ...
2
votes
2answers
260 views

How to compare strings in two files

I have two text files, one file contains entries such as Id Value 1 apple 2 orange 3 mango 4 banana 5 strawberry 6 papaya In other ...
1
vote
1answer
56 views

In-place substitution for paste command [duplicate]

When I run the following code echo '1' > file.txt echo '2' | paste file.txt - > file.txt I would expect the content of file.txt be 1 2, because the paste command as it name suggests should ...
4
votes
2answers
143 views

how to sort by the day of the week?

I have two files. file1: Dave 734.838.9800 Bob 313.123.4567 Carol 248.344.5576 Mary 313.449.1390 Ted 248.496.2204 Alice 616.556.4458 file2: Bob Tuesday Carol Monday Ted Sunday ...
1
vote
1answer
98 views

How to merge two files in the same row?

I have two files. file1: Dave 734.838.9800 Bob 313.123.4567 Carol 248.344.5576 Mary 313.449.1390 Ted 248.496.2204 Alice 616.556.4458 file2: Bob Tuesday Carol Monday Ted Sunday ...
3
votes
1answer
36 views

Find repository names from gitolite info output

How do I extract the names from the gitolite info command output, for further piping into a script? I'm writing a migration script to migrate all my repositories from gitolite to a Gitlab server. ...
2
votes
2answers
63 views

Need to obtain Awk output in specific sequence

Data in file: 1234567-5678907 3456789-1234563 3456789-1234567 . . . . n Now I want the output in the below format: 1234567-5678907,3456789-1234563,3456789-1234563....n I have used below command ...
2
votes
1answer
75 views

ksh/bash Formatting Files through sed by cat file|sed command

I am new to unix platform, (Alternate Approaches can be discussed) I am trying to change the colors of content of a file conditionally. My file is: A B C D Value Value Value Value Value ...
1
vote
2answers
30 views

How to merge the lines in Unix (Sun Solaris) in a specific pattern

I have below n number of lines in a flat file: uk,1234560000 uk,6789067000 uk,4567890000 . . . n lines Now I have to make it in below format: ...
1
vote
1answer
48 views

If statement in shell-script only half-working

I've been trying to write a script to rotate my monitor, I'll then assign it to a keyboard key. I've been fairly successful in making it rotate once, however when I run the script again, it doesn't ...
1
vote
1answer
64 views

Cut some fields from a text file to another

I have the following command: grep -owf champs.txt t.txt and I want to cut the result of this command from the file t.txt to another file. I know that it can be achieved by a nested command but I ...