Manipulation of text by programs, scripts, etc.

learn more… | top users | synonyms

4
votes
6answers
121 views

Sed — Replace first k instances of a word in the file

I want to replace only the first k instances of a word. How can I do this? Eg. Say file foo.txt contains 100 instances occurrences of word 'linux' . I need to replace first 50 occurrences only.
2
votes
2answers
11 views

Replace line names with set of new names without disturbing the rest of the file

I have a large protein sequence file where each sequence name is identified using > and the corresponding sequence in the following lines. Example (ignore quotation marks): >YAL003W EFB1 ...
6
votes
2answers
242 views

Compare two files and print matches

Is there anyway to get this through Unix shell scripting? I have a fileA with one column (1000 rows), and fileB with 26 columns(13000 rows). I need to search each value of fileA with fileB and return ...
8
votes
3answers
887 views

How do I take a list and remove it from a file?

I have a long list of domain names that I need to remove from /etc/remotedomains. They're probably not in any particular order in the file. Each domain is on one line. How could I iterate through the ...
1
vote
0answers
17 views

Create files from templates, substitutions in middle of other words

Is there a way to convince m4 to replace a macro in the middle of a word? I would like this file, day.m4: define(Sat,Cat)dnl Saturday and this command: m4 day.m4 to produce this output: ...
2
votes
3answers
40 views

Merging two tables including multiple ocurrence of column identifiers and unique lines

I asked a similar question yesterday (Merging two tables including multiple ocurrence of column identifiers) but ran into a problem with unique lines. I would like to merge two tables based on column ...
1
vote
3answers
35 views

Replace word instances with its count attached to it

I have a file which contains word 'name' multiple times ( 250 times ). I need to replace every instance of 'name' with its count attached to it as name_1, name_2, name_3....name_250. Can I do this ...
1
vote
2answers
42 views

Count the maximum character length for all the data fields in a csv file and output to txt

Given a CSV with many data fields (>50), how can I count the maximum character length for each data field and then export all the counts to a txt file? BTW, I want to ignore the first line of the file ...
0
votes
0answers
22 views

Shell Script for tracking number of times each command executes

How to have a shell script that takes input from /var/log/sudo.log and sends results to a spreadsheet that displays graph for each command execution? I want to record how many times each command ...
2
votes
2answers
92 views

Merging two tables including multiple ocurrence of column identifiers

I would like to merge two tables: File 1: 1 today 2 tomorrow 3 red File 2: 1 a lot 1 sometimes 2 at work 2 at home 2 sometimes 3 new Desired output (file 3): 1 today ...
0
votes
1answer
285 views

How to print gmail inbox messages

I want to print inbox messages one by one (like cat someFile) so that I can parse the message contents. something like printGmail InboxMesg1 | grep some | sed some Using curl I can access the ...
3
votes
2answers
174 views

awk: Extracting a fixed number of rows where the last row number may vary

This question may look like a duplicate, but only at first glance. Of course, I would no longer need help in how to code a one-liner that extracts a fixed number of continuous lines (e. g. 5 in this ...
0
votes
2answers
26 views

How to find the occurrence of names in a file?

I want to know how many times users who have logged on the system in a single command. I have used repetitive commands to find the how many times he have logged. $ last | grep -c user1 $ last | grep ...
1
vote
1answer
40 views

Use case for /usr/bin/lessecho

I can not come up with an example of how I could/would use lessecho. The description in the man page is fairly straight forward but is not helping me come up with a use case: DESCRIPTION ...
1
vote
3answers
40 views

sed match pattern N times

I want to split a file into chunks with 2 words each. $cat tmp word1 word2 word3 word4 word5 word6 word7 $sed -e 's/word. word. /&\n/g' tmp word1 word2 word3 word4 word5 word6 word7 $sed -e ...
1
vote
1answer
41 views

Store output from one command and process it for another

I am working on Unix command line. I have two files. I want to cat file1.txt and grep the output in file2.txt File1.txt 123A 223A 143A 153A 183A 123J 123P File2.txt: 123A (TYU) 223A (RUT) 143A ...
1
vote
4answers
146 views

Decimal conversion number of the form x.xx to 0x.xx, sed?

I have a file of the form ... num 0.12 num num num 25.53 num num num 7.82 num num ... and I want to convert the 2nd field of each line adding a "0" at the numbers >= 0 and < 10 so the output ...
-2
votes
2answers
52 views

Grep and cut the selected field

I am trying to cut the field if a line starts with a particular name, cat $1|while read line do if [ "$1" = "custbills.cmp" ]; then acc_no=`grep "^Custbills" $1 | cut -c29-43` acc_type=`grep ...
-2
votes
1answer
35 views

Place the cut field in the desired position of ouput file [closed]

I am writing a script like in which, it will cut the fields from input file and append to output file. cat $1|while read line do acc_no=`echo "$line" | cut -c29-43` acc_type=`echo "$line"|cut ...
5
votes
7answers
633 views

What is the easiest way to add a string on the beginning of every line of the file from the command line?

I am looking for a way to add some string to the beginning of every line (same string for every line). Not something customizable but rather something that will be easy to remember and available on ...
2
votes
3answers
134 views

Text File Renaming Using Script

I have thousands of text files inside a folder and they are named in a certain way. For example: Image_234_Data_7778_n0.txt Image_234_Data_7778_n1.txt Image_234_Data_7778_n2.txt ...
0
votes
4answers
634 views

How to get data count between two strings with multiple occurences in the same file

I have the data in my file as START-OF-FIELDS ID NAME DEPT END-OF-FIELDS START-OF-DATA 1|joy|cs 2|sam|ec END-OF-DATA START-OF-FIELDS ID NAME DOB DEPT ADDRESS END-OF-FIELDS START-OF-DATA ...
2
votes
3answers
60 views

how to concatenate a file with multiple header into one

Can you suggest me how I can combine. a.fasta with 10000 sequences having headers 1, 2, 3, .... 10000 .......................into just one header e.g >1 AAATTTTGGGGCCC >2 ACCCCGGGTTT ...
2
votes
0answers
17 views

Make GNU indent break long comments

Is there a way to convince GNU indent to break long comment and change it to multiline comment? Something like this: // Very long comment, longer than 80 characters. Just imagine that. To this: /* ...
1
vote
4answers
47 views

Parsing complex text file using Unix commands

I have the following text structure which I would like to parse: >Cluster 423 0 56aa, >HWI-ST1448:257:C3V2HACXX:1:1106:19087:2550.1... at 92.86% 1 64aa, ...
1
vote
4answers
68 views

Bash one liner to change configuration parameters

I have a config file with following structure. ValueOne = 1 ValueTwo = 2 ValueThree = 3 I want a one liner bash script to find ValueTwo and change the value to 22222. Any idea? Not School Thing
1
vote
1answer
24 views

What does “position” mean in iconv error message?

I get error when converting a large file: $ iconv -f GB2312 -t UTF-8 2001.txt -o 2001_u.txt iconv: illegal input sequence at position 245256667 What does the position mean in this error? I tried it ...
5
votes
5answers
83 views

Replace data at specific positions in txt file using data from another file

I have a text file in the below format: $data This is the experimental data good data This is good file datafile 1 4324 3673 6.2e+11 7687 67576 2 3565 8768 8760 5780 8778 "This is line ...
1
vote
3answers
64 views

Print content of file from desired character and end also to desired character

I have this file with content written below. I want output of my file stating with ".SUBCKT C032_THS_COMPARATOR" and end with ".ENDS": .SUBCKT C032_THS_DPDNPROT_cdm_tdpdnw TOPROTECT VPLUS .SUBCKT ...
0
votes
4answers
46 views

Adding a character to every other text line

I have a huge (ca. 20G) text file which contains millions of passages (a.k.a. paragraphs) with headers. Headers and paragraphs are always one line each, e.g., Sunshine This is a sunny day. ...
3
votes
3answers
108 views

Finding groups of IP addresses in a file

I have a file with groups of IP addresses. The file looks like this: London: 1.1.1.0-1.1.1.200 172.25.2.0-172.25.2.100 Germany: 2.2.2.0-2.2.2.100 192.168.1.0-192.168.1.200 172.25.2.0-172.25.2.200 ...
5
votes
3answers
71 views

How to use awk through multiple files?

I want a solution for the general case (N folder) I'm using awk to process a file and extract its content and put it in a variable then echo it: This is the file: H1 H2 H3 H4 H5 H6 H7 H8 H9 not ...
3
votes
3answers
141 views

Replacing a pattern in line preceding another pattern of interest

I have a log file with many instances of statements as shown below: 14 Aug 19 16:30:11.506: <DATA> |POS|IDLE|1|01131844090|5950|$hostIp|$size |$data 20 Aug 19 16:30:12.439: <DATA> ...
1
vote
5answers
99 views

Is there a way to find out the number of lines from a command output?

I am trying to compare command outputs between two servers. I want to find out the number of lines from the output of commands such as ls -l. Is there a way to do so? So far I have not found anything. ...
7
votes
1answer
347 views

Is there a way to make perl -i not clobber symlinks?

A friend of mine points out that if you do: perl -pi.bak -e 's/foo/bar/' somefile when "somefile" is actually a symlink, perl does just what the docs say it will do: It does this by renaming ...
21
votes
2answers
9k views

How can I make iconv replace the input file with the converted output?

I have a bash script which enumerates through every *.php file in a directory and applies iconv to it. This gets output in STDOUT. Since adding the -o parameter ( in my experience ) actually writes ...
0
votes
2answers
47 views

Sum up the selected fields

I have a file with a numeric value in character positions 86 to 99. I want to sum up the entire amount present in the file, only for lines which contain RS at position 124-125. I am trying to do like ...
3
votes
2answers
77 views

Extract specific set lines matching a rule

I have a large file with the logs as shown below. There are about 30000 instance of such events logged. I need to extract those lines beginning with RINGING and CLOSE (included) and which does NOT ...
4
votes
4answers
63 views

stream edit >GDRLR5L04EQY8D 3931|1166 into >3931

I have large text DNA sequence files with the following format >name AACTGGGCT I need to parse the name line which currently has extra information and special characters, while keeping any line ...
3
votes
2answers
51 views

Compare 2 tab delimited files and output differences with column header

I would like to compare 2 similar files on a common column. The files will have identical headers. file1.txt mem_id Date Time Building aa bb cc dd ee ff ...
1
vote
3answers
55 views

How to convert html entities to readable text?

I want html number entities like &#x119; and want to convert it to real character. I have emails mostly from linkedin that look like this: chcia&#x142;abym zapyta&#x107;, czy ...
3
votes
3answers
85 views

Determine maximum column length for every column in csv-file

To determine the maximum length of each column in a comma-separated csv-file I hacked together a bash-script. When I ran it on a linux system it produced the correct output, but I need it to run on OS ...
1
vote
2answers
31 views

Error: ambiguous redirect while transferring output to command

Output to redirect in script: 511@ubuntu:~/Unix/test$ ls -ltr |awk '{print $9}' default.txt dfah.txt fruit.txt fruit_prices.txt dfh.txt header.txt testfile.txt topoutput.txt Script written on shell ...
1
vote
3answers
41 views

cat a directories files apending the file name to the row of text and removing the header row? [closed]

How can I cat a directories files apending the file name to the row of text and removing the header row? So as to only have 1 header row in the all.txt file.
12
votes
1answer
3k views

How can I replace a string in a file(s)?

Replacing strings in files based on certain search criteria is a very common task. How can I replace string foo with bar in all files in the current directory? do the same recursively for sub ...
4
votes
1answer
40 views

Truncated result returned by JOIN

I'm using join command under linux, but the results vary between different machines. I have two simple files: cat 1.txt a aaa,0.2 b bbb,0.3 c ccc,0.5 cat 2.txt a aaa,0.2 b bbb,0.3 c ccc,0.6 I'm ...
1
vote
4answers
27 views

Loop through array

I have records in list.txt (INPUT) as List of animals SET 1=Dog 2=Crow 3=Snake Pet,bird,reptile List of Countries SET 1=France 2=Singapore 3=Columbia Europe,Asia,SouthAmerica List of Parts SET ...
5
votes
7answers
511 views

How to find a position of a character using grep?

I need to identify the postion of a character in string using grep command. Example, the string is RAMSITALSKHMAN|1223333. grep -n '[^a-zA-Z0-9\$\~\%\#\^]' How do I find the position of | in the ...
0
votes
1answer
52 views

Extract values from HTML Files

I have 2 HTML files, where part of the content looks like this: In FILE1: <td width="48%" align="right" valign="top"> <b>mom. Wirkleistung P+ tot.: </b><br> <b>mom. ...
1
vote
1answer
51 views

Why is the output of this paste command split on separate lines with indents?

I was trying to merge files using either of these commands: paste data_{1..2}.txt > 1_2.txt paste <(awk '{print $1}' data_2.txt ) <(awk '{print $1}' data_2.txt) > 1_2.txt But the ...