Manipulation of text by programs, scripts, etc.

learn more… | top users | synonyms

3
votes
2answers
86 views

Extract specific set lines matching a rule

I have a large file with the logs as shown below. There are about 30000 instance of such events logged. I need to extract those lines beginning with RINGING and CLOSE (included) and which does NOT ...
0
votes
2answers
54 views

Sum up the selected fields

I have a file with a numeric value in character positions 86 to 99. I want to sum up the entire amount present in the file, only for lines which contain RS at position 124-125. I am trying to do like ...
4
votes
4answers
66 views

stream edit >GDRLR5L04EQY8D 3931|1166 into >3931

I have large text DNA sequence files with the following format >name AACTGGGCT I need to parse the name line which currently has extra information and special characters, while keeping any line ...
1
vote
2answers
138 views

Error: ambiguous redirect while transferring output to command

Output to redirect in script: 511@ubuntu:~/Unix/test$ ls -ltr |awk '{print $9}' default.txt dfah.txt fruit.txt fruit_prices.txt dfh.txt header.txt testfile.txt topoutput.txt Script written on shell ...
3
votes
2answers
69 views

Compare 2 tab delimited files and output differences with column header

I would like to compare 2 similar files on a common column. The files will have identical headers. file1.txt mem_id Date Time Building aa bb cc dd ee ff ...
1
vote
3answers
56 views

cat a directories files apending the file name to the row of text and removing the header row? [closed]

How can I cat a directories files apending the file name to the row of text and removing the header row? So as to only have 1 header row in the all.txt file.
3
votes
3answers
152 views

Determine maximum column length for every column in csv-file

To determine the maximum length of each column in a comma-separated csv-file I hacked together a bash-script. When I ran it on a linux system it produced the correct output, but I need it to run on OS ...
4
votes
1answer
44 views

Truncated result returned by JOIN

I'm using join command under linux, but the results vary between different machines. I have two simple files: cat 1.txt a aaa,0.2 b bbb,0.3 c ccc,0.5 cat 2.txt a aaa,0.2 b bbb,0.3 c ccc,0.6 I'm ...
1
vote
4answers
32 views

Loop through array

I have records in list.txt (INPUT) as List of animals SET 1=Dog 2=Crow 3=Snake Pet,bird,reptile List of Countries SET 1=France 2=Singapore 3=Columbia Europe,Asia,SouthAmerica List of Parts SET ...
3
votes
3answers
241 views

Get the part of a line before the last slash

I'm trying to write a conky script that shows my MPD album art, a 'folder.jpg' in the album folder. My current plan is to use mpc -f %file%, which prints out the file name and path, and then cut out ...
1
vote
1answer
63 views

Why is the output of this paste command split on separate lines with indents?

I was trying to merge files using either of these commands: paste data_{1..2}.txt > 1_2.txt paste <(awk '{print $1}' data_2.txt ) <(awk '{print $1}' data_2.txt) > 1_2.txt But the ...
5
votes
7answers
683 views

How to find a position of a character using grep?

I need to identify the postion of a character in string using grep command. Example, the string is RAMSITALSKHMAN|1223333. grep -n '[^a-zA-Z0-9\$\~\%\#\^]' How do I find the position of | in the ...
4
votes
3answers
599 views

Replace “o” with “0” in large wordlist and save the original word?

I have a large text file with words. Every word is on a single line (typical wordlist). I want to replace all characters "o" with the number "0" in every word, and the new formed word to be pasted on ...
0
votes
1answer
60 views

Extract values from HTML Files

I have 2 HTML files, where part of the content looks like this: In FILE1: <td width="48%" align="right" valign="top"> <b>mom. Wirkleistung P+ tot.: </b><br> <b>mom. ...
3
votes
3answers
128 views

Finding groups of IP addresses in a file

I have a file with groups of IP addresses. The file looks like this: London: 1.1.1.0-1.1.1.200 172.25.2.0-172.25.2.100 Germany: 2.2.2.0-2.2.2.100 192.168.1.0-192.168.1.200 172.25.2.0-172.25.2.200 ...
1
vote
1answer
28 views

What does “position” mean in iconv error message?

I get error when converting a large file: $ iconv -f GB2312 -t UTF-8 2001.txt -o 2001_u.txt iconv: illegal input sequence at position 245256667 What does the position mean in this error? I tried it ...
2
votes
4answers
79 views

Filter Require Capital words from a file( not all capital words )

I want Output of this file only AVDDPLL1V8 AGNDPLL1V8 DVDDPLL1V1 DGNDPLL1V1 Please help me in this to find solution. Here is my input: 6.1.2 Power and Ground Pins The following ...
0
votes
1answer
61 views

How to copy rows of different files and paste column-wise?

I want to create a new file by copying rows of each file and pasting columnwise according to their file name orders. Here is the sample input and required output files below. file1.txt 1234 5678 ...
1
vote
3answers
82 views

Pulling lines n to m from list of files found

Ultimately I need to grab lines 3-53 from ever csv in each subdirectory. I grabbed the lines like so (cat isn't necessarily required): cat /[path]/[file].csv | head -53 | tail -51 And the files ...
-1
votes
1answer
49 views

To append columns from file 1 to 3 for each row that the first cell of that row matches ow create a new row and then append

I have four CSV files as examples given below; I want to append columns from file 1 to 3 for each row that the first cell of that row matches ow create a new row and then append. file1: ...
0
votes
5answers
114 views

print only rows that has latest non empty cell that starts with the word “Failed” on a CSV file in Linux [closed]

I want to print the table only if non-empty latest column that starts with the word "Failed"on a CSV file in Linux.. My input - Name,Mon,Tue,Wed,Thu,Fri,Mon Linux,Successfull Fullbackup; Failed ...
2
votes
4answers
99 views

sed load substitute from file

I'm attempting to replace a pattern in a file with the complete contents of another file using sed. Currently I am using: sed "s/PATTERN/`cat replacement.txt`/g" "openfile.txt" > "output.txt" ...
2
votes
1answer
41 views

Command to print few consecutive lines from middle of a file [duplicate]

For a file containing 20 lines, lines 6-10 can be printed using following command: head -10 filename | tail -5 Can this exactly same thing be done without using 'head' and 'tail' commands ?? ...
2
votes
2answers
25 views

How to escape a Whole Variable?

In the answer of thisquestion: https://superuser.com/questions/163515/bash-how-to-pass-command-line-arguments-containing-special-characters It said that we have to put an arg of a function in ...
3
votes
3answers
50 views

Removal of lines with more than 'N' fields?

I am working on mac with sed, perl, awk, bash.. I have a large-ish (10GB) text file which has 13 fields (columns) of TAB delimited data. Unfortunately some of these lines have extraneous TABs , so I ...
3
votes
1answer
123 views

Tokenize string from $REPLY in bash script

This is my first post, and I have no idea how I managed anything before StackExchage, Google, Wiki, GNU, Internet, the list goes on :) I am trying to find a way to construct a SQL database generator ...
3
votes
2answers
42 views

Extract text between two specific lines

How can I get the part of the output of a command between two specific lines? A dummy example: Command: git for-each-ref --sort='*authordate' --format='%(tag)' refs/tags | grep -v '^$' Output: ...
0
votes
1answer
70 views

Add text to end of every line

I have tried a lot of commands, like this: sed -i '' "s/$/<@string>/" txt.txt > txt2.txt This makes txt2.txt empty, why?!? sed 's/$/<@string>/' txt.txt > txt2.txt This does ...
8
votes
6answers
260 views

Obtain values of a section of a file

I have a configuration file in the following format. <Title> [part1] A.File = a A.Val = val1 B.File = a B.Val = val1 [part2] A.File = a1 A.Val = val2 B.File = a B.Val = val1 ...
9
votes
3answers
569 views

Why does “wc -c” print a value of one more with echo?

When running echo abcd | wc -c it returns 5. But the word abcd is only 4 characters long. Is echo printing some special character after the word abcd? And can I prevent echo from printing that? ...
2
votes
4answers
179 views

Iteratively edit a single file

I have a single file with a statement that looks like this : myName_tx_1 VARCHAR(255) myName_in_1 VARCHAR(255) myName_tx_2 VARCHAR(255) myName_in_2 VARCHAR(255) myName_tx_3 VARCHAR(255) myName_in_3 ...
1
vote
5answers
107 views

How to overwrite stdout using echo? [closed]

We know to overwrite bar.txt, we can use this: echo "foo" > bar.txt but how can we overwrite to the standard output? I tried this command: echo "foo" > but it failed. I want this because ...
3
votes
2answers
125 views

Remove duplicate fields in a given column (after ID check)

I'd like to remove from a given column ($2 in the example) the duplicate fields (comma separated), but only if the ID column ($1 in the example) is the same. Input: A 1,2,3,4 A 8,9,10,11 A ...
3
votes
3answers
129 views

Parse multiple sections of data into seperate files

I have a concatenated log file with multiple logs inside that I'm trying to parse out into individual log files (I will later rename them to the date/time of each). Each log is separated by "--- LOG ...
2
votes
2answers
27 views

removing each . after ] appear in a line in complete text file

I have a file in which I need to eliminate . after the ] on every line e.g xyz[1.2]35.23 xyz[1.2.3]35.234 output should be xyz[1.2]3523 xyz[1.2.3]35234
1
vote
2answers
28 views

Transform custom tags with arguments

I want to write my diploma theses with markdown. Sadly, there is no perfect editor and so I have to improvise. For bibtex citations I want to transform my own definition with some sort of bash-filter ...
3
votes
2answers
295 views

Get text between a word and the last line?

According to the answers in Stack Overflow question How to use sed/grep to extract text between two words?, we can get text between two words: sed -n "/first/,/second/p" file But what if I want to ...
2
votes
1answer
43 views

Calculate Apache's response time for last 10 seconds

I have the next apache's log format: LogFormat "%h %l %u %t \"%r\" %>s %b %D \"%{Referer}i\" \"%{User-Agent}i\"" where's %D - The time taken to serve the request, in microseconds. I want to ...
0
votes
4answers
71 views

How print only the current logged in “root” user IP address?

Currently I can see my (root) IP address on top of the output for this command: last But what I want is just that the IP address is printed out to me in a clean way. root pts/1 1.1.1.1 ...
1
vote
3answers
76 views

Trouble in adding a column with awk

I'm trying to add a column to a csv file by using: awk -F "," '{print $0"new_col"}' file.csv but I got : new_col8,0.405,0.101 new_col4,0.374,0.100 new_col23,0.342,0.100 new_col130,0.298,0.100 ...
5
votes
4answers
456 views

How can I “lazily” read output from xrandr?

I have a bash script that I use to adjust my monitor brightness that uses xrandr --verbose to get the current brightness. It works fine, but using xrandr is kind of slow on my machine, as you can see ...
5
votes
4answers
229 views

How to replace all the first occurences in matched lines with incremental number?

I have a file like this ... 1562 first part 1563 H col3 H col4 1564 H col3 H col4 ... 3241 H col3 H col4 3242 third part ... I want to replace only the first H in every line to H#, with ...
1
vote
2answers
99 views

How do I validate a settings(ini) file for the values expected using bash?

After I check if the ini file exists where it is expected to be, which I've learned how to do, I would like to check the file for proper formatting. I'd like to validate any number of "site" entries ...
0
votes
1answer
27 views

Compare strings

I have 2 txt files, in one i have alias lists and the other is the servers name. So what im trying to do is to loop from the alias txt file line by line and where it will find the name that is on the ...
2
votes
1answer
31 views

Get each column which contains multiple spaces?

My file: subnet 172.25.10.0 netmask 255.255.255.0 { option domain-name-servers 172.25.10.9, 8.8.8.8; default-lease-time 300; max-lease-time 7500; range dynamic-bootp 172.25.10.10 172.25.10.30 ; ...
-2
votes
3answers
62 views

How to sed two same value without affecting the other

I have an xml file and I want to change one value on that file: param name=swam_license_ports_overdraft_thresh 99 param name="strike_license_ports_overdraft_thresh" 99 The outcome I want is to ...
3
votes
2answers
97 views

Identify lines that are out of order

I have a process that generates output mostly in lexicographically sorted order according to a (timestamp) field, but occasionally the lines will be output in the wrong order: 2014-08-14 ...
4
votes
6answers
296 views

want the last line when duplicate awk?

I have the file: key value blah blah blah blah blahblah man1 boy1 blah blah man1 boy2 man1 boy1 I do this to remove duplicate lines: awk '/man1/ { print $1,$2} ' file | awk '!x[$0]++' and the ...
1
vote
3answers
123 views

Joining files on disk

Say I have several file with the following naming: 000001 000002 000003 ... Each of these files is a csv file (may include escape characters). In total the folder has ~20GB of data. How can I ...
3
votes
2answers
143 views

How to count how many different word do i have in a file, in a column

I have a CSV file. "AGNOLI Valerio","ITA","AST" In this example, the 2nd column says "ITA". I expect there are about 100 or so different nationalities listed in this file. I want to know exactly ...