Manipulation of text by programs, scripts, etc.

learn more… | top users | synonyms

0
votes
2answers
37 views

Finding groups of IP addresses in a file

I have some files with groups of IP addresses. The file looks like this: London: 1.1.1.0-1.1.1.200 172.25.2.0-172.25.2.100 Germany: 2.2.2.0-2.2.2.100 192.168.1.0-192.168.1.200 172.25.2.0-172.25.2.200 ...
1
vote
1answer
12 views

What does “position” mean in iconv error message?

I get error when converting a large file: $ iconv -f GB2312 -t UTF-8 2001.txt -o 2001_u.txt iconv: illegal input sequence at position 245256667 What does the position mean in this error? I tried it ...
2
votes
4answers
43 views

Filter Require Capital words from a file( not all capital words )

I want Output of this file only AVDDPLL1V8 AGNDPLL1V8 DVDDPLL1V1 DGNDPLL1V1 Please help me in this to find solution. Here is my input: 6.1.2 Power and Ground Pins The following ...
0
votes
1answer
37 views

How to copy rows of different files and paste column-wise?

I want to create a new file by copying rows of each file and pasting columnwise according to their file name orders. Here is the sample input and required output files below. file1.txt 1234 5678 ...
-1
votes
1answer
42 views

To append columns from file 1 to 3 for each row that the first cell of that row matches ow create a new row and then append

I have four CSV files as examples given below; I want to append columns from file 1 to 3 for each row that the first cell of that row matches ow create a new row and then append. file1: ...
0
votes
5answers
69 views

print only rows that has latest non empty cell that starts with the word “Failed” on a CSV file in Linux [on hold]

I want to print the table only if non-empty latest column that starts with the word "Failed"on a CSV file in Linux.. My input - Name,Mon,Tue,Wed,Thu,Fri,Mon Linux,Successfull Fullbackup; Failed ...
2
votes
4answers
83 views

sed load substitute from file

I'm attempting to replace a pattern in a file with the complete contents of another file using sed. Currently I am using: sed "s/PATTERN/`cat replacement.txt`/g" "openfile.txt" > "output.txt" ...
2
votes
1answer
26 views

Command to print few consecutive lines from middle of a file [duplicate]

For a file containing 20 lines, lines 6-10 can be printed using following command: head -10 filename | tail -5 Can this exactly same thing be done without using 'head' and 'tail' commands ?? ...
-3
votes
0answers
46 views

please let me know where to attach the files here? [closed]

Below is the requirement. Ignore lines containing pattern "SET SNAP_ID" or "Shadow Copy Components:\" . Ignore lines with column 2 till 8 is empty. print lines containing pattern "Failed". ...
2
votes
2answers
22 views

How to escape a Whole Variable?

In the answer of thisquestion: https://superuser.com/questions/163515/bash-how-to-pass-command-line-arguments-containing-special-characters It said that we have to put an arg of a function in ...
3
votes
3answers
47 views

Removal of lines with more than 'N' fields?

I am working on mac with sed, perl, awk, bash.. I have a large-ish (10GB) text file which has 13 fields (columns) of TAB delimited data. Unfortunately some of these lines have extraneous TABs , so I ...
3
votes
1answer
70 views

Tokenize string from $REPLY in bash script

This is my first post, and I have no idea how I managed anything before StackExchage, Google, Wiki, GNU, Internet, the list goes on :) I am trying to find a way to construct a SQL database generator ...
3
votes
2answers
30 views

Extract text between two specific lines

How can I get the part of the output of a command between two specific lines? A dummy example: Command: git for-each-ref --sort='*authordate' --format='%(tag)' refs/tags | grep -v '^$' Output: ...
0
votes
1answer
42 views

Add text to end of every line

I have tried a lot of commands, like this: sed -i '' "s/$/<@string>/" txt.txt > txt2.txt This makes txt2.txt empty, why?!? sed 's/$/<@string>/' txt.txt > txt2.txt This does ...
6
votes
6answers
221 views

Obtain values of a section of a file

I have a configuration file in the following format. <Title> [part1] A.File = a A.Val = val1 B.File = a B.Val = val1 [part2] A.File = a1 A.Val = val2 B.File = a B.Val = val1 ...
9
votes
3answers
524 views

Why does “wc -c” print a value of one more with echo?

When running echo abcd | wc -c it returns 5. But the word abcd is only 4 characters long. Is echo printing some special character after the word abcd? And can I prevent echo from printing that? ...
2
votes
4answers
172 views

Iteratively edit a single file

I have a single file with a statement that looks like this : myName_tx_1 VARCHAR(255) myName_in_1 VARCHAR(255) myName_tx_2 VARCHAR(255) myName_in_2 VARCHAR(255) myName_tx_3 VARCHAR(255) myName_in_3 ...
1
vote
5answers
63 views

How to overwrite stdout using echo? [closed]

We know to overwrite bar.txt, we can use this: echo "foo" > bar.txt but how can we overwrite to the standard output? I tried this command: echo "foo" > but it failed. I want this because ...
3
votes
2answers
117 views

Remove duplicate fields in a given column (after ID check)

I'd like to remove from a given column ($2 in the example) the duplicate fields (comma separated), but only if the ID column ($1 in the example) is the same. Input: A 1,2,3,4 A 8,9,10,11 A ...
3
votes
3answers
119 views

Parse multiple sections of data into seperate files

I have a concatenated log file with multiple logs inside that I'm trying to parse out into individual log files (I will later rename them to the date/time of each). Each log is separated by "--- LOG ...
2
votes
2answers
27 views

removing each . after ] appear in a line in complete text file

I have a file in which I need to eliminate . after the ] on every line e.g xyz[1.2]35.23 xyz[1.2.3]35.234 output should be xyz[1.2]3523 xyz[1.2.3]35234
1
vote
2answers
26 views

Transform custom tags with arguments

I want to write my diploma theses with markdown. Sadly, there is no perfect editor and so I have to improvise. For bibtex citations I want to transform my own definition with some sort of bash-filter ...
-3
votes
0answers
25 views

Parsing the log file test into tabular format

How to display below test file in tabular format as below: Output should be like: ma1-test1.cor.dns.com cmd - ...
2
votes
2answers
209 views

Get text between a word and the last line?

According to the answers in Stack Overflow question How to use sed/grep to extract text between two words?, we can get text between two words: sed -n "/first/,/second/p" file But what if I want to ...
2
votes
1answer
28 views

Calculate Apache's response time for last 10 seconds

I have the next apache's log format: LogFormat "%h %l %u %t \"%r\" %>s %b %D \"%{Referer}i\" \"%{User-Agent}i\"" where's %D - The time taken to serve the request, in microseconds. I want to ...
0
votes
4answers
65 views

How print only the current logged in “root” user IP address?

Currently I can see my (root) IP address on top of the output for this command: last But what I want is just that the IP address is printed out to me in a clean way. root pts/1 1.1.1.1 ...
1
vote
3answers
39 views

Trouble in adding a column with awk

I'm trying to add a column to a csv file by using: awk -F "," '{print $0"new_col"}' file.csv but I got : new_col8,0.405,0.101 new_col4,0.374,0.100 new_col23,0.342,0.100 new_col130,0.298,0.100 ...
5
votes
4answers
426 views

How can I “lazily” read output from xrandr?

I have a bash script that I use to adjust my monitor brightness that uses xrandr --verbose to get the current brightness. It works fine, but using xrandr is kind of slow on my machine, as you can see ...
5
votes
4answers
210 views

How to replace all the first occurences in matched lines with incremental number?

I have a file like this ... 1562 first part 1563 H col3 H col4 1564 H col3 H col4 ... 3241 H col3 H col4 3242 third part ... I want to replace only the first H in every line to H#, with ...
0
votes
2answers
43 views

How do I validate a settings(ini) file for the values expected using bash?

After I check if the ini file exists where it is expected to be, which I've learned how to do, I would like to check the file for proper formatting. I'd like to validate any number of "site" entries ...
0
votes
1answer
22 views

Compare strings

I have 2 txt files, in one i have alias lists and the other is the servers name. So what im trying to do is to loop from the alias txt file line by line and where it will find the name that is on the ...
2
votes
1answer
23 views

Get each column which contains multiple spaces?

My file: subnet 172.25.10.0 netmask 255.255.255.0 { option domain-name-servers 172.25.10.9, 8.8.8.8; default-lease-time 300; max-lease-time 7500; range dynamic-bootp 172.25.10.10 172.25.10.30 ; ...
-2
votes
3answers
59 views

How to sed two same value without affecting the other

I have an xml file and I want to change one value on that file: param name=swam_license_ports_overdraft_thresh 99 param name="strike_license_ports_overdraft_thresh" 99 The outcome I want is to ...
3
votes
2answers
95 views

Identify lines that are out of order

I have a process that generates output mostly in lexicographically sorted order according to a (timestamp) field, but occasionally the lines will be output in the wrong order: 2014-08-14 ...
4
votes
6answers
285 views

want the last line when duplicate awk?

I have the file: key value blah blah blah blah blahblah man1 boy1 blah blah man1 boy2 man1 boy1 I do this to remove duplicate lines: awk '/man1/ { print $1,$2} ' file | awk '!x[$0]++' and the ...
1
vote
3answers
121 views

Joining files on disk

Say I have several file with the following naming: 000001 000002 000003 ... Each of these files is a csv file (may include escape characters). In total the folder has ~20GB of data. How can I ...
3
votes
2answers
138 views

How to count how many different word do i have in a file, in a column

I have a CSV file. "AGNOLI Valerio","ITA","AST" In this example, the 2nd column says "ITA". I expect there are about 100 or so different nationalities listed in this file. I want to know exactly ...
3
votes
1answer
43 views

Handling dynamically changing column positions and splitting file

I have a unique problem. I have a file with following columns PersonId, AB1, AB2, AB3, DG1_B1,DG1_B2,DG2 1,0,1,0,3,5,7 2,0,3,1,3,7,4 I want to split this and create multiple files based on the ...
-2
votes
2answers
74 views

Text processing using unix tools

I have a file in the following format. abc|xyz|mno x up x up up x x x up abc|xyz|mno x x x up x x .... .... Basically the first line of the file are device names (|ed) and ...
2
votes
1answer
35 views

Why might these values not be uniq'ed together?

I am trying to figure out which IP addresses hit the server most often. I received this surprising output from uniq: $ cat access_log | awk '{print $1}' | uniq -c | sort -bnr | head 26 73.23.5.25 ...
4
votes
6answers
151 views

grep with a lot of files which contains a lot of lines?

Imagine we have a lot of huge(30000 line)files: a.2014-05-06, a.2014-05-07, a.2014-05-08 and so on. I know that we can grep a line with word using this command: grep "word" a.* First I guess this ...
2
votes
4answers
74 views

Insert new line after non matching pattern

I have following output from script host1:101 host1:102 host2:101 host2:102 host2:103 host4:101 host5:101 host5:102 host5:103 host5:104 etc.. I wanted to insert a new line after each host group. ...
3
votes
3answers
84 views

How can I delete text before a delimiter in Linux

I need to delete text from a line before delimiter. For example: (123434): hello::{apple,orange,mango}. I need to delete text before first :. i.e. (123434). Is there any command in linux to ...
1
vote
3answers
31 views

Increment a date in a file

I have a file with single line of text inside.How can I modify its content without using the vi editor.Below is the example: PROCESS DATE =2014-08-01 I want to increment the "2014-08-01" by the ...
1
vote
2answers
65 views

Text File Renaming Using Script

I have thousands of text files inside a folder and they are named in a certain way. For example: Image_234_Data_7778_n0.txt Image_234_Data_7778_n1.txt Image_234_Data_7778_n2.txt ...
2
votes
6answers
426 views

Handling comma in string values in a CSV file [duplicate]

I have a comma-separated file which has numeric and string columns. String columns are quoted and can have comma in between the quotes. How do I identify the columns with FS =","? Sample records ...
1
vote
3answers
68 views

merging all files into one file?

I was trying to merge all the text files in to one file. All the text files has one column and equal number of rows. My data file names are data_1.txt, data_2.txt, data_3.txt, . . . data_n.txt. ...
5
votes
2answers
79 views

Extract tomcat parameters from ps output

I'm trying to write a little diagnostic tool and one of the functions I want from it is that I want to run is: ps axu | grep tomcat | grep -v grep | awk '{ print $<list of desired fields> }' ...
0
votes
2answers
100 views

Delete every number that comes before “=”

i have a very, very, very long list. Like over 830,000 characters. I will of course perform said command on a copy. There are multiple lines, like this hey1=lol hey2=lol hey3=lol I want that to ...
2
votes
2answers
108 views

Collate tab-delimited files

I want to compare the information from mapping files to retrieve crosslinked information in a tab-delimited file format. For example, one file contains COG0001 882.DVU3168 COG0002 883.DvMF_2502 ...