Tagged Questions

Manipulation of text by programs, scripts, etc.

learn more… | top users | synonyms

-1
votes
3answers
118 views

How can I select columns based on a control file with invalid column numbers?

I have to generate the file dynamically from the source file based on the below control file. control_file.txt 1,3,5,-1,8,-1,4 The control file contains the position of columns which I require ...
6
votes
2answers
282 views

Compare two files and print matches

Is there anyway to get this through Unix shell scripting? I have a fileA with one column (1000 rows), and fileB with 26 columns(13000 rows). I need to search each value of fileA with fileB and return ...
1
vote
0answers
27 views

Create files from templates, substitutions in middle of other words

Is there a way to convince m4 to replace a macro in the middle of a word? I would like this file, day.m4: define(Sat,Cat)dnl Saturday and this command: m4 day.m4 to produce this output: ...
2
votes
3answers
47 views

Merging two tables including multiple ocurrence of column identifiers and unique lines

I asked a similar question yesterday (Merging two tables including multiple ocurrence of column identifiers) but ran into a problem with unique lines. I would like to merge two tables based on column ...
1
vote
3answers
49 views

Replace word instances with its count attached to it

I have a file which contains word 'name' multiple times ( 250 times ). I need to replace every instance of 'name' with its count attached to it as name_1, name_2, name_3....name_250. Can I do this ...
4
votes
6answers
162 views

Sed — Replace first k instances of a word in the file

I want to replace only the first k instances of a word. How can I do this? Eg. Say file foo.txt contains 100 instances occurrences of word 'linux' . I need to replace first 50 occurrences only.
2
votes
2answers
108 views

Merging two tables including multiple ocurrence of column identifiers

I would like to merge two tables: File 1: 1 today 2 tomorrow 3 red File 2: 1 a lot 1 sometimes 2 at work 2 at home 2 sometimes 3 new Desired output (file 3): 1 today ...
1
vote
2answers
63 views

Count the maximum character length for all the data fields in a csv file and output to txt

Given a CSV with many data fields (>50), how can I count the maximum character length for each data field and then export all the counts to a txt file? BTW, I want to ignore the first line of the file ...
0
votes
0answers
24 views

Shell Script for tracking number of times each command executes

How to have a shell script that takes input from /var/log/sudo.log and sends results to a spreadsheet that displays graph for each command execution? I want to record how many times each command ...
0
votes
2answers
27 views

How to find the occurrence of names in a file?

I want to know how many times users who have logged on the system in a single command. I have used repetitive commands to find the how many times he have logged. $ last | grep -c user1 $ last | grep ...
1
vote
3answers
46 views

sed match pattern N times

I want to split a file into chunks with 2 words each. $cat tmp word1 word2 word3 word4 word5 word6 word7 $sed -e 's/word. word. /&\n/g' tmp word1 word2 word3 word4 word5 word6 word7 $sed -e ...
1
vote
1answer
51 views

Use case for /usr/bin/lessecho

I can not come up with an example of how I could/would use lessecho. The description in the man page is fairly straight forward but is not helping me come up with a use case: DESCRIPTION ...
1
vote
4answers
166 views

Decimal conversion number of the form x.xx to 0x.xx, sed?

I have a file of the form ... num 0.12 num num num 25.53 num num num 7.82 num num ... and I want to convert the 2nd field of each line adding a "0" at the numbers >= 0 and < 10 so the output ...
-2
votes
1answer
40 views

Place the cut field in the desired position of ouput file [closed]

I am writing a script like in which, it will cut the fields from input file and append to output file. cat $1|while read line do acc_no=`echo "$line" | cut -c29-43` acc_type=`echo "$line"|cut ...
-1
votes
1answer
38 views

convert scientific data into generic

I have a file containing only numerical values in scientific format. I want to convert them into generic notation and multiply them by 1000. e.g. input.txt 1.1234e-01 2.123e+01 3.12345e-02 ... ...
1
vote
1answer
48 views

Store output from one command and process it for another

I am working on Unix command line. I have two files. I want to cat file1.txt and grep the output in file2.txt File1.txt 123A 223A 143A 153A 183A 123J 123P File2.txt: 123A (TYU) 223A (RUT) 143A ...
1
vote
4answers
61 views

Parsing complex text file using Unix commands

I have the following text structure which I would like to parse: >Cluster 423 0 56aa, >HWI-ST1448:257:C3V2HACXX:1:1106:19087:2550.1... at 92.86% 1 64aa, ...
2
votes
3answers
74 views

how to concatenate a file with multiple header into one

Can you suggest me how I can combine. a.fasta with 10000 sequences having headers 1, 2, 3, .... 10000 .......................into just one header e.g >1 AAATTTTGGGGCCC >2 ACCCCGGGTTT ...
2
votes
0answers
22 views

Make GNU indent break long comments

Is there a way to convince GNU indent to break long comment and change it to multiline comment? Something like this: // Very long comment, longer than 80 characters. Just imagine that. To this: /* ...
3
votes
2answers
197 views

awk: Extracting a fixed number of rows where the last row number may vary

This question may look like a duplicate, but only at first glance. Of course, I would no longer need help in how to code a one-liner that extracts a fixed number of continuous lines (e. g. 5 in this ...
1
vote
3answers
70 views

Print content of file from desired character and end also to desired character

I have this file with content written below. I want output of my file stating with ".SUBCKT C032_THS_COMPARATOR" and end with ".ENDS": .SUBCKT C032_THS_DPDNPROT_cdm_tdpdnw TOPROTECT VPLUS .SUBCKT ...
-2
votes
2answers
78 views

Grep and cut the selected field

I am trying to cut the field if a line starts with a particular name, cat $1|while read line do if [ "$1" = "custbills.cmp" ]; then acc_no=`grep "^Custbills" $1 | cut -c29-43` acc_type=`grep ...
5
votes
5answers
108 views

Replace data at specific positions in txt file using data from another file

I have a text file in the below format: $data This is the experimental data good data This is good file datafile 1 4324 3673 6.2e+11 7687 67576 2 3565 8768 8760 5780 8778 "This is line ...
3
votes
3answers
143 views

Replacing a pattern in line preceding another pattern of interest

I have a log file with many instances of statements as shown below: 14 Aug 19 16:30:11.506: <DATA> |POS|IDLE|1|01131844090|5950|$hostIp|$size |$data 20 Aug 19 16:30:12.439: <DATA> ...
5
votes
3answers
113 views

How to use awk through multiple files?

I want a solution for the general case (N folder) I'm using awk to process a file and extract its content and put it in a variable then echo it: This is the file: H1 H2 H3 H4 H5 H6 H7 H8 H9 not ...
0
votes
4answers
61 views

Adding a character to every other text line

I have a huge (ca. 20G) text file which contains millions of passages (a.k.a. paragraphs) with headers. Headers and paragraphs are always one line each, e.g., Sunshine This is a sunny day. ...
3
votes
2answers
87 views

Extract specific set lines matching a rule

I have a large file with the logs as shown below. There are about 30000 instance of such events logged. I need to extract those lines beginning with RINGING and CLOSE (included) and which does NOT ...
0
votes
2answers
54 views

Sum up the selected fields

I have a file with a numeric value in character positions 86 to 99. I want to sum up the entire amount present in the file, only for lines which contain RS at position 124-125. I am trying to do like ...
4
votes
4answers
66 views

stream edit >GDRLR5L04EQY8D 3931|1166 into >3931

I have large text DNA sequence files with the following format >name AACTGGGCT I need to parse the name line which currently has extra information and special characters, while keeping any line ...
1
vote
2answers
188 views

Error: ambiguous redirect while transferring output to command

Output to redirect in script: 511@ubuntu:~/Unix/test$ ls -ltr |awk '{print $9}' default.txt dfah.txt fruit.txt fruit_prices.txt dfh.txt header.txt testfile.txt topoutput.txt Script written on shell ...
3
votes
2answers
79 views

Compare 2 tab delimited files and output differences with column header

I would like to compare 2 similar files on a common column. The files will have identical headers. file1.txt mem_id Date Time Building aa bb cc dd ee ff ...
1
vote
3answers
56 views

cat a directories files apending the file name to the row of text and removing the header row? [closed]

How can I cat a directories files apending the file name to the row of text and removing the header row? So as to only have 1 header row in the all.txt file.
3
votes
3answers
192 views

Determine maximum column length for every column in csv-file

To determine the maximum length of each column in a comma-separated csv-file I hacked together a bash-script. When I ran it on a linux system it produced the correct output, but I need it to run on OS ...
4
votes
1answer
44 views

Truncated result returned by JOIN

I'm using join command under linux, but the results vary between different machines. I have two simple files: cat 1.txt a aaa,0.2 b bbb,0.3 c ccc,0.5 cat 2.txt a aaa,0.2 b bbb,0.3 c ccc,0.6 I'm ...
1
vote
4answers
35 views

Loop through array

I have records in list.txt (INPUT) as List of animals SET 1=Dog 2=Crow 3=Snake Pet,bird,reptile List of Countries SET 1=France 2=Singapore 3=Columbia Europe,Asia,SouthAmerica List of Parts SET ...
3
votes
3answers
256 views

Get the part of a line before the last slash

I'm trying to write a conky script that shows my MPD album art, a 'folder.jpg' in the album folder. My current plan is to use mpc -f %file%, which prints out the file name and path, and then cut out ...
1
vote
1answer
66 views

Why is the output of this paste command split on separate lines with indents?

I was trying to merge files using either of these commands: paste data_{1..2}.txt > 1_2.txt paste <(awk '{print $1}' data_2.txt ) <(awk '{print $1}' data_2.txt) > 1_2.txt But the ...
5
votes
7answers
780 views

How to find a position of a character using grep?

I need to identify the postion of a character in string using grep command. Example, the string is RAMSITALSKHMAN|1223333. grep -n '[^a-zA-Z0-9\$\~\%\#\^]' How do I find the position of | in the ...
4
votes
3answers
599 views

Replace “o” with “0” in large wordlist and save the original word?

I have a large text file with words. Every word is on a single line (typical wordlist). I want to replace all characters "o" with the number "0" in every word, and the new formed word to be pasted on ...
0
votes
1answer
62 views

Extract values from HTML Files

I have 2 HTML files, where part of the content looks like this: In FILE1: <td width="48%" align="right" valign="top"> <b>mom. Wirkleistung P+ tot.: </b><br> <b>mom. ...
3
votes
3answers
135 views

Finding groups of IP addresses in a file

I have a file with groups of IP addresses. The file looks like this: London: 1.1.1.0-1.1.1.200 172.25.2.0-172.25.2.100 Germany: 2.2.2.0-2.2.2.100 192.168.1.0-192.168.1.200 172.25.2.0-172.25.2.200 ...
1
vote
1answer
29 views

What does “position” mean in iconv error message?

I get error when converting a large file: $ iconv -f GB2312 -t UTF-8 2001.txt -o 2001_u.txt iconv: illegal input sequence at position 245256667 What does the position mean in this error? I tried it ...
2
votes
4answers
82 views

Filter Require Capital words from a file( not all capital words )

I want Output of this file only AVDDPLL1V8 AGNDPLL1V8 DVDDPLL1V1 DGNDPLL1V1 Please help me in this to find solution. Here is my input: 6.1.2 Power and Ground Pins The following ...
0
votes
1answer
64 views

How to copy rows of different files and paste column-wise?

I want to create a new file by copying rows of each file and pasting columnwise according to their file name orders. Here is the sample input and required output files below. file1.txt 1234 5678 ...
1
vote
3answers
82 views

Pulling lines n to m from list of files found

Ultimately I need to grab lines 3-53 from ever csv in each subdirectory. I grabbed the lines like so (cat isn't necessarily required): cat /[path]/[file].csv | head -53 | tail -51 And the files ...
-1
votes
1answer
50 views

To append columns from file 1 to 3 for each row that the first cell of that row matches ow create a new row and then append

I have four CSV files as examples given below; I want to append columns from file 1 to 3 for each row that the first cell of that row matches ow create a new row and then append. file1: ...
0
votes
5answers
116 views

print only rows that has latest non empty cell that starts with the word “Failed” on a CSV file in Linux [closed]

I want to print the table only if non-empty latest column that starts with the word "Failed"on a CSV file in Linux.. My input - Name,Mon,Tue,Wed,Thu,Fri,Mon Linux,Successfull Fullbackup; Failed ...
2
votes
4answers
102 views

sed load substitute from file

I'm attempting to replace a pattern in a file with the complete contents of another file using sed. Currently I am using: sed "s/PATTERN/`cat replacement.txt`/g" "openfile.txt" > "output.txt" ...
2
votes
1answer
41 views

Command to print few consecutive lines from middle of a file [duplicate]

For a file containing 20 lines, lines 6-10 can be printed using following command: head -10 filename | tail -5 Can this exactly same thing be done without using 'head' and 'tail' commands ?? ...
2
votes
2answers
25 views

How to escape a Whole Variable?

In the answer of thisquestion: https://superuser.com/questions/163515/bash-how-to-pass-command-line-arguments-containing-special-characters It said that we have to put an arg of a function in ...