Manipulation of text by programs, scripts, etc.

learn more… | top users | synonyms

-2
votes
2answers
74 views

Grep and cut the selected field

I am trying to cut the field if a line starts with a particular name, cat $1|while read line do if [ "$1" = "custbills.cmp" ]; then acc_no=`grep "^Custbills" $1 | cut -c29-43` acc_type=`grep ...
5
votes
5answers
103 views

Replace data at specific positions in txt file using data from another file

I have a text file in the below format: $data This is the experimental data good data This is good file datafile 1 4324 3673 6.2e+11 7687 67576 2 3565 8768 8760 5780 8778 "This is line ...
3
votes
3answers
143 views

Replacing a pattern in line preceding another pattern of interest

I have a log file with many instances of statements as shown below: 14 Aug 19 16:30:11.506: <DATA> |POS|IDLE|1|01131844090|5950|$hostIp|$size |$data 20 Aug 19 16:30:12.439: <DATA> ...
5
votes
3answers
105 views

How to use awk through multiple files?

I want a solution for the general case (N folder) I'm using awk to process a file and extract its content and put it in a variable then echo it: This is the file: H1 H2 H3 H4 H5 H6 H7 H8 H9 not ...
0
votes
4answers
60 views

Adding a character to every other text line

I have a huge (ca. 20G) text file which contains millions of passages (a.k.a. paragraphs) with headers. Headers and paragraphs are always one line each, e.g., Sunshine This is a sunny day. ...
3
votes
2answers
87 views

Extract specific set lines matching a rule

I have a large file with the logs as shown below. There are about 30000 instance of such events logged. I need to extract those lines beginning with RINGING and CLOSE (included) and which does NOT ...
0
votes
2answers
54 views

Sum up the selected fields

I have a file with a numeric value in character positions 86 to 99. I want to sum up the entire amount present in the file, only for lines which contain RS at position 124-125. I am trying to do like ...
4
votes
4answers
66 views

stream edit >GDRLR5L04EQY8D 3931|1166 into >3931

I have large text DNA sequence files with the following format >name AACTGGGCT I need to parse the name line which currently has extra information and special characters, while keeping any line ...
1
vote
2answers
151 views

Error: ambiguous redirect while transferring output to command

Output to redirect in script: 511@ubuntu:~/Unix/test$ ls -ltr |awk '{print $9}' default.txt dfah.txt fruit.txt fruit_prices.txt dfh.txt header.txt testfile.txt topoutput.txt Script written on shell ...
3
votes
2answers
69 views

Compare 2 tab delimited files and output differences with column header

I would like to compare 2 similar files on a common column. The files will have identical headers. file1.txt mem_id Date Time Building aa bb cc dd ee ff ...
1
vote
3answers
56 views

cat a directories files apending the file name to the row of text and removing the header row? [closed]

How can I cat a directories files apending the file name to the row of text and removing the header row? So as to only have 1 header row in the all.txt file.
3
votes
3answers
167 views

Determine maximum column length for every column in csv-file

To determine the maximum length of each column in a comma-separated csv-file I hacked together a bash-script. When I ran it on a linux system it produced the correct output, but I need it to run on OS ...
4
votes
1answer
44 views

Truncated result returned by JOIN

I'm using join command under linux, but the results vary between different machines. I have two simple files: cat 1.txt a aaa,0.2 b bbb,0.3 c ccc,0.5 cat 2.txt a aaa,0.2 b bbb,0.3 c ccc,0.6 I'm ...
1
vote
4answers
33 views

Loop through array

I have records in list.txt (INPUT) as List of animals SET 1=Dog 2=Crow 3=Snake Pet,bird,reptile List of Countries SET 1=France 2=Singapore 3=Columbia Europe,Asia,SouthAmerica List of Parts SET ...
3
votes
3answers
245 views

Get the part of a line before the last slash

I'm trying to write a conky script that shows my MPD album art, a 'folder.jpg' in the album folder. My current plan is to use mpc -f %file%, which prints out the file name and path, and then cut out ...
1
vote
1answer
66 views

Why is the output of this paste command split on separate lines with indents?

I was trying to merge files using either of these commands: paste data_{1..2}.txt > 1_2.txt paste <(awk '{print $1}' data_2.txt ) <(awk '{print $1}' data_2.txt) > 1_2.txt But the ...
5
votes
7answers
719 views

How to find a position of a character using grep?

I need to identify the postion of a character in string using grep command. Example, the string is RAMSITALSKHMAN|1223333. grep -n '[^a-zA-Z0-9\$\~\%\#\^]' How do I find the position of | in the ...
4
votes
3answers
599 views

Replace “o” with “0” in large wordlist and save the original word?

I have a large text file with words. Every word is on a single line (typical wordlist). I want to replace all characters "o" with the number "0" in every word, and the new formed word to be pasted on ...
0
votes
1answer
60 views

Extract values from HTML Files

I have 2 HTML files, where part of the content looks like this: In FILE1: <td width="48%" align="right" valign="top"> <b>mom. Wirkleistung P+ tot.: </b><br> <b>mom. ...
3
votes
3answers
132 views

Finding groups of IP addresses in a file

I have a file with groups of IP addresses. The file looks like this: London: 1.1.1.0-1.1.1.200 172.25.2.0-172.25.2.100 Germany: 2.2.2.0-2.2.2.100 192.168.1.0-192.168.1.200 172.25.2.0-172.25.2.200 ...
1
vote
1answer
29 views

What does “position” mean in iconv error message?

I get error when converting a large file: $ iconv -f GB2312 -t UTF-8 2001.txt -o 2001_u.txt iconv: illegal input sequence at position 245256667 What does the position mean in this error? I tried it ...
2
votes
4answers
80 views

Filter Require Capital words from a file( not all capital words )

I want Output of this file only AVDDPLL1V8 AGNDPLL1V8 DVDDPLL1V1 DGNDPLL1V1 Please help me in this to find solution. Here is my input: 6.1.2 Power and Ground Pins The following ...
0
votes
1answer
61 views

How to copy rows of different files and paste column-wise?

I want to create a new file by copying rows of each file and pasting columnwise according to their file name orders. Here is the sample input and required output files below. file1.txt 1234 5678 ...
1
vote
3answers
82 views

Pulling lines n to m from list of files found

Ultimately I need to grab lines 3-53 from ever csv in each subdirectory. I grabbed the lines like so (cat isn't necessarily required): cat /[path]/[file].csv | head -53 | tail -51 And the files ...
-1
votes
1answer
50 views

To append columns from file 1 to 3 for each row that the first cell of that row matches ow create a new row and then append

I have four CSV files as examples given below; I want to append columns from file 1 to 3 for each row that the first cell of that row matches ow create a new row and then append. file1: ...
0
votes
5answers
115 views

print only rows that has latest non empty cell that starts with the word “Failed” on a CSV file in Linux [closed]

I want to print the table only if non-empty latest column that starts with the word "Failed"on a CSV file in Linux.. My input - Name,Mon,Tue,Wed,Thu,Fri,Mon Linux,Successfull Fullbackup; Failed ...
2
votes
4answers
101 views

sed load substitute from file

I'm attempting to replace a pattern in a file with the complete contents of another file using sed. Currently I am using: sed "s/PATTERN/`cat replacement.txt`/g" "openfile.txt" > "output.txt" ...
2
votes
1answer
41 views

Command to print few consecutive lines from middle of a file [duplicate]

For a file containing 20 lines, lines 6-10 can be printed using following command: head -10 filename | tail -5 Can this exactly same thing be done without using 'head' and 'tail' commands ?? ...
2
votes
2answers
25 views

How to escape a Whole Variable?

In the answer of thisquestion: https://superuser.com/questions/163515/bash-how-to-pass-command-line-arguments-containing-special-characters It said that we have to put an arg of a function in ...
3
votes
3answers
50 views

Removal of lines with more than 'N' fields?

I am working on mac with sed, perl, awk, bash.. I have a large-ish (10GB) text file which has 13 fields (columns) of TAB delimited data. Unfortunately some of these lines have extraneous TABs , so I ...
3
votes
1answer
127 views

Tokenize string from $REPLY in bash script

This is my first post, and I have no idea how I managed anything before StackExchage, Google, Wiki, GNU, Internet, the list goes on :) I am trying to find a way to construct a SQL database generator ...
3
votes
2answers
42 views

Extract text between two specific lines

How can I get the part of the output of a command between two specific lines? A dummy example: Command: git for-each-ref --sort='*authordate' --format='%(tag)' refs/tags | grep -v '^$' Output: ...
0
votes
1answer
72 views

Add text to end of every line

I have tried a lot of commands, like this: sed -i '' "s/$/<@string>/" txt.txt > txt2.txt This makes txt2.txt empty, why?!? sed 's/$/<@string>/' txt.txt > txt2.txt This does ...
8
votes
6answers
262 views

Obtain values of a section of a file

I have a configuration file in the following format. <Title> [part1] A.File = a A.Val = val1 B.File = a B.Val = val1 [part2] A.File = a1 A.Val = val2 B.File = a B.Val = val1 ...
9
votes
3answers
570 views

Why does “wc -c” print a value of one more with echo?

When running echo abcd | wc -c it returns 5. But the word abcd is only 4 characters long. Is echo printing some special character after the word abcd? And can I prevent echo from printing that? ...
2
votes
4answers
179 views

Iteratively edit a single file

I have a single file with a statement that looks like this : myName_tx_1 VARCHAR(255) myName_in_1 VARCHAR(255) myName_tx_2 VARCHAR(255) myName_in_2 VARCHAR(255) myName_tx_3 VARCHAR(255) myName_in_3 ...
1
vote
5answers
111 views

How to overwrite stdout using echo? [closed]

We know to overwrite bar.txt, we can use this: echo "foo" > bar.txt but how can we overwrite to the standard output? I tried this command: echo "foo" > but it failed. I want this because ...
3
votes
2answers
127 views

Remove duplicate fields in a given column (after ID check)

I'd like to remove from a given column ($2 in the example) the duplicate fields (comma separated), but only if the ID column ($1 in the example) is the same. Input: A 1,2,3,4 A 8,9,10,11 A ...
3
votes
3answers
131 views

Parse multiple sections of data into seperate files

I have a concatenated log file with multiple logs inside that I'm trying to parse out into individual log files (I will later rename them to the date/time of each). Each log is separated by "--- LOG ...
2
votes
2answers
28 views

removing each . after ] appear in a line in complete text file

I have a file in which I need to eliminate . after the ] on every line e.g xyz[1.2]35.23 xyz[1.2.3]35.234 output should be xyz[1.2]3523 xyz[1.2.3]35234
1
vote
2answers
28 views

Transform custom tags with arguments

I want to write my diploma theses with markdown. Sadly, there is no perfect editor and so I have to improvise. For bibtex citations I want to transform my own definition with some sort of bash-filter ...
3
votes
2answers
305 views

Get text between a word and the last line?

According to the answers in Stack Overflow question How to use sed/grep to extract text between two words?, we can get text between two words: sed -n "/first/,/second/p" file But what if I want to ...
2
votes
1answer
46 views

Calculate Apache's response time for last 10 seconds

I have the next apache's log format: LogFormat "%h %l %u %t \"%r\" %>s %b %D \"%{Referer}i\" \"%{User-Agent}i\"" where's %D - The time taken to serve the request, in microseconds. I want to ...
0
votes
4answers
71 views

How print only the current logged in “root” user IP address?

Currently I can see my (root) IP address on top of the output for this command: last But what I want is just that the IP address is printed out to me in a clean way. root pts/1 1.1.1.1 ...
1
vote
3answers
85 views

Trouble in adding a column with awk

I'm trying to add a column to a csv file by using: awk -F "," '{print $0"new_col"}' file.csv but I got : new_col8,0.405,0.101 new_col4,0.374,0.100 new_col23,0.342,0.100 new_col130,0.298,0.100 ...
5
votes
4answers
462 views

How can I “lazily” read output from xrandr?

I have a bash script that I use to adjust my monitor brightness that uses xrandr --verbose to get the current brightness. It works fine, but using xrandr is kind of slow on my machine, as you can see ...
5
votes
4answers
230 views

How to replace all the first occurences in matched lines with incremental number?

I have a file like this ... 1562 first part 1563 H col3 H col4 1564 H col3 H col4 ... 3241 H col3 H col4 3242 third part ... I want to replace only the first H in every line to H#, with ...
1
vote
2answers
101 views

How do I validate a settings(ini) file for the values expected using bash?

After I check if the ini file exists where it is expected to be, which I've learned how to do, I would like to check the file for proper formatting. I'd like to validate any number of "site" entries ...
0
votes
1answer
27 views

Compare strings

I have 2 txt files, in one i have alias lists and the other is the servers name. So what im trying to do is to loop from the alias txt file line by line and where it will find the name that is on the ...
2
votes
1answer
31 views

Get each column which contains multiple spaces?

My file: subnet 172.25.10.0 netmask 255.255.255.0 { option domain-name-servers 172.25.10.9, 8.8.8.8; default-lease-time 300; max-lease-time 7500; range dynamic-bootp 172.25.10.10 172.25.10.30 ; ...