Tagged Questions

Manipulation of text by programs, scripts, etc.

learn more… | top users | synonyms

1
vote
1answer
50 views

How to Replace IP at Specific Line

I have a file (.htaccess) which contains below excerpt. <Files wp-login.php> order deny,allow deny from all allow from 45.152.35.2 </Files> I need to replace 45.152.35.2 with my current ...
4
votes
4answers
170 views

How to remove all comments from a file?

I have a file with a few lines commented out: foo bar stuff #Do not show this... morestuff evenmorestuff#Or this I just want to print all the uncommented code: foo bar stuff morestuff ...
0
votes
1answer
22 views

filter fileds in a file, without constant record format

I have 3 files with millions of records each. I'd like to filter those files in order to find some data more easily. Unfortunately, there is no common format from one record to other. For example: ...
1
vote
4answers
110 views

how to match the number with % from df -k command

in my bash script I want to print the use capacity of /var partition from df -k command I do the following df -k | grep var | awk '{print $5}' the problem is that sometimes "use%" number ...
0
votes
2answers
43 views

Add space after 8 characters then add space after every 4 characters

Here is the sample input: AbbigailAbieAbbyAbbi Using sed, I have separated them into 4 characters but seems like I got a problem because I have to separate the first 8 characters first then add ...
-1
votes
1answer
68 views

Match and combine records in Bash [duplicate]

Consider two csv files: $ cat given.csv 123,John 246,Paul 369,George 987,Ringo $ cat family.csv 246,McCartney 123,Lennon 987,Starr 369,Harrison Note that the lines are not ordered! How might I use ...
2
votes
5answers
125 views

Extract lines from a file containg From <str1> To <str2>

Let's says there is file "abc.txt" that contains a paragraph/lines: Hello, how are you doing sir? when are you going to arrive at the SBAHN platform to catch the train? How can I extract all the ...
2
votes
1answer
38 views

Comparing delimited fields

I have two fields in a file, students and teachers. They're delimited by a semicolon and I want to find which single students have two different teachers. jdoe;ateacher jdoe;bteacher jsmith;cteacher ...
0
votes
3answers
85 views

Combinatorics of letters and words - preferably bash, will accept perl or other [closed]

I am trying to write a shell script to generate all possible words in the English language less than 20 characters. I doubt there is any truly efficient way to do this other than to brute force some ...
0
votes
2answers
56 views

Convert two rows to two columns

I would like to convert every two rows to two columns using awk. Any help appreciated. input.txt: # Query: gi|11465907|ref|NC_001872.1| Chlamydomonas eugametos genome, complete genome # 0 hits found ...
0
votes
3answers
148 views

bash find using variable

I have an issue I don't understand. It is simple and it should work, but it doesn't. =EDITED exactly what I can see from terminal= I have a list of filenames: [molni@archlinux picasa_album]$ cat LIST ...
2
votes
3answers
122 views

Use AWK to pad date and time formats

I would like to use AWK to pad the date and time fields in my report. Here is my current .csv that is actually tab delimited. AA 9/7/2014 2:30:38 PM AA0000011111 08 Y A Jane, Doe AA ...
2
votes
2answers
20 views

Replace line names with set of new names without disturbing the rest of the file

I have a large protein sequence file where each sequence name is identified using > and the corresponding sequence in the following lines. Example (ignore quotation marks): >YAL003W EFB1 ...
-1
votes
3answers
149 views

How can I select columns based on a control file with invalid column numbers?

I have to generate the file dynamically from the source file based on the below control file. control_file.txt 1,3,5,-1,8,-1,4 The control file contains the position of columns which I require ...
6
votes
2answers
290 views

Compare two files and print matches

Is there anyway to get this through Unix shell scripting? I have a fileA with one column (1000 rows), and fileB with 26 columns(13000 rows). I need to search each value of fileA with fileB and return ...
1
vote
0answers
28 views

Create files from templates, substitutions in middle of other words

Is there a way to convince m4 to replace a macro in the middle of a word? I would like this file, day.m4: define(Sat,Cat)dnl Saturday and this command: m4 day.m4 to produce this output: ...
2
votes
3answers
49 views

Merging two tables including multiple ocurrence of column identifiers and unique lines

I asked a similar question yesterday (Merging two tables including multiple ocurrence of column identifiers) but ran into a problem with unique lines. I would like to merge two tables based on column ...
1
vote
3answers
51 views

Replace word instances with its count attached to it

I have a file which contains word 'name' multiple times ( 250 times ). I need to replace every instance of 'name' with its count attached to it as name_1, name_2, name_3....name_250. Can I do this ...
4
votes
6answers
192 views

Sed — Replace first k instances of a word in the file

I want to replace only the first k instances of a word. How can I do this? Eg. Say file foo.txt contains 100 instances occurrences of word 'linux' . I need to replace first 50 occurrences only.
2
votes
2answers
113 views

Merging two tables including multiple ocurrence of column identifiers

I would like to merge two tables: File 1: 1 today 2 tomorrow 3 red File 2: 1 a lot 1 sometimes 2 at work 2 at home 2 sometimes 3 new Desired output (file 3): 1 today ...
1
vote
2answers
81 views

Count the maximum character length for all the data fields in a csv file and output to txt

Given a CSV with many data fields (>50), how can I count the maximum character length for each data field and then export all the counts to a txt file? BTW, I want to ignore the first line of the file ...
0
votes
0answers
25 views

Shell Script for tracking number of times each command executes

How to have a shell script that takes input from /var/log/sudo.log and sends results to a spreadsheet that displays graph for each command execution? I want to record how many times each command ...
0
votes
2answers
28 views

How to find the occurrence of names in a file?

I want to know how many times users who have logged on the system in a single command. I have used repetitive commands to find the how many times he have logged. $ last | grep -c user1 $ last | grep ...
1
vote
3answers
49 views

sed match pattern N times

I want to split a file into chunks with 2 words each. $cat tmp word1 word2 word3 word4 word5 word6 word7 $sed -e 's/word. word. /&\n/g' tmp word1 word2 word3 word4 word5 word6 word7 $sed -e ...
1
vote
1answer
57 views

Use case for /usr/bin/lessecho

I can not come up with an example of how I could/would use lessecho. The description in the man page is fairly straight forward but is not helping me come up with a use case: DESCRIPTION ...
1
vote
4answers
176 views

Decimal conversion number of the form x.xx to 0x.xx, sed?

I have a file of the form ... num 0.12 num num num 25.53 num num num 7.82 num num ... and I want to convert the 2nd field of each line adding a "0" at the numbers >= 0 and < 10 so the output ...
-2
votes
1answer
40 views

Place the cut field in the desired position of ouput file [closed]

I am writing a script like in which, it will cut the fields from input file and append to output file. cat $1|while read line do acc_no=`echo "$line" | cut -c29-43` acc_type=`echo "$line"|cut ...
-1
votes
1answer
38 views

convert scientific data into generic

I have a file containing only numerical values in scientific format. I want to convert them into generic notation and multiply them by 1000. e.g. input.txt 1.1234e-01 2.123e+01 3.12345e-02 ... ...
1
vote
1answer
51 views

Store output from one command and process it for another

I am working on Unix command line. I have two files. I want to cat file1.txt and grep the output in file2.txt File1.txt 123A 223A 143A 153A 183A 123J 123P File2.txt: 123A (TYU) 223A (RUT) 143A ...
1
vote
4answers
70 views

Parsing complex text file using Unix commands

I have the following text structure which I would like to parse: >Cluster 423 0 56aa, >HWI-ST1448:257:C3V2HACXX:1:1106:19087:2550.1... at 92.86% 1 64aa, ...
2
votes
3answers
83 views

how to concatenate a file with multiple header into one

Can you suggest me how I can combine. a.fasta with 10000 sequences having headers 1, 2, 3, .... 10000 .......................into just one header e.g >1 AAATTTTGGGGCCC >2 ACCCCGGGTTT ...
2
votes
0answers
25 views

Make GNU indent break long comments

Is there a way to convince GNU indent to break long comment and change it to multiline comment? Something like this: // Very long comment, longer than 80 characters. Just imagine that. To this: /* ...
3
votes
2answers
210 views

awk: Extracting a fixed number of rows where the last row number may vary

This question may look like a duplicate, but only at first glance. Of course, I would no longer need help in how to code a one-liner that extracts a fixed number of continuous lines (e. g. 5 in this ...
1
vote
3answers
72 views

Print content of file from desired character and end also to desired character

I have this file with content written below. I want output of my file stating with ".SUBCKT C032_THS_COMPARATOR" and end with ".ENDS": .SUBCKT C032_THS_DPDNPROT_cdm_tdpdnw TOPROTECT VPLUS .SUBCKT ...
-2
votes
2answers
106 views

Grep and cut the selected field

I am trying to cut the field if a line starts with a particular name, cat $1|while read line do if [ "$1" = "custbills.cmp" ]; then acc_no=`grep "^Custbills" $1 | cut -c29-43` acc_type=`grep ...
5
votes
5answers
146 views

Replace data at specific positions in txt file using data from another file

I have a text file in the below format: $data This is the experimental data good data This is good file datafile 1 4324 3673 6.2e+11 7687 67576 2 3565 8768 8760 5780 8778 "This is line ...
3
votes
3answers
143 views

Replacing a pattern in line preceding another pattern of interest

I have a log file with many instances of statements as shown below: 14 Aug 19 16:30:11.506: <DATA> |POS|IDLE|1|01131844090|5950|$hostIp|$size |$data 20 Aug 19 16:30:12.439: <DATA> ...
5
votes
3answers
126 views

How to use awk through multiple files?

I want a solution for the general case (N folder) I'm using awk to process a file and extract its content and put it in a variable then echo it: This is the file: H1 H2 H3 H4 H5 H6 H7 H8 H9 not ...
0
votes
4answers
83 views

Adding a character to every other text line

I have a huge (ca. 20G) text file which contains millions of passages (a.k.a. paragraphs) with headers. Headers and paragraphs are always one line each, e.g., Sunshine This is a sunny day. ...
3
votes
2answers
88 views

Extract specific set lines matching a rule

I have a large file with the logs as shown below. There are about 30000 instance of such events logged. I need to extract those lines beginning with RINGING and CLOSE (included) and which does NOT ...
0
votes
2answers
58 views

Sum up the selected fields

I have a file with a numeric value in character positions 86 to 99. I want to sum up the entire amount present in the file, only for lines which contain RS at position 124-125. I am trying to do like ...
4
votes
4answers
67 views

stream edit >GDRLR5L04EQY8D 3931|1166 into >3931

I have large text DNA sequence files with the following format >name AACTGGGCT I need to parse the name line which currently has extra information and special characters, while keeping any line ...
1
vote
2answers
365 views

Error: ambiguous redirect while transferring output to command

Output to redirect in script: 511@ubuntu:~/Unix/test$ ls -ltr |awk '{print $9}' default.txt dfah.txt fruit.txt fruit_prices.txt dfh.txt header.txt testfile.txt topoutput.txt Script written on shell ...
3
votes
2answers
93 views

Compare 2 tab delimited files and output differences with column header

I would like to compare 2 similar files on a common column. The files will have identical headers. file1.txt mem_id Date Time Building aa bb cc dd ee ff ...
1
vote
3answers
63 views

cat a directories files apending the file name to the row of text and removing the header row? [closed]

How can I cat a directories files apending the file name to the row of text and removing the header row? So as to only have 1 header row in the all.txt file.
3
votes
3answers
253 views

Determine maximum column length for every column in csv-file

To determine the maximum length of each column in a comma-separated csv-file I hacked together a bash-script. When I ran it on a linux system it produced the correct output, but I need it to run on OS ...
4
votes
1answer
45 views

Truncated result returned by JOIN

I'm using join command under linux, but the results vary between different machines. I have two simple files: cat 1.txt a aaa,0.2 b bbb,0.3 c ccc,0.5 cat 2.txt a aaa,0.2 b bbb,0.3 c ccc,0.6 I'm ...
1
vote
4answers
42 views

Loop through array

I have records in list.txt (INPUT) as List of animals SET 1=Dog 2=Crow 3=Snake Pet,bird,reptile List of Countries SET 1=France 2=Singapore 3=Columbia Europe,Asia,SouthAmerica List of Parts SET ...
3
votes
3answers
299 views

Get the part of a line before the last slash

I'm trying to write a conky script that shows my MPD album art, a 'folder.jpg' in the album folder. My current plan is to use mpc -f %file%, which prints out the file name and path, and then cut out ...
1
vote
1answer
72 views

Why is the output of this paste command split on separate lines with indents?

I was trying to merge files using either of these commands: paste data_{1..2}.txt > 1_2.txt paste <(awk '{print $1}' data_2.txt ) <(awk '{print $1}' data_2.txt) > 1_2.txt But the ...