Manipulation of text by programs, scripts, etc.

learn more… | top users | synonyms

5
votes
6answers
241 views

How to delete every second line from a file?

File: Data inserted into table. Total count 13 No error occurred Data inserted into table. Total count 45 No error occurred Data inserted into table. Total count 14 No error occurred Data inserted ...
-1
votes
1answer
36 views

How to add a word at the first of every lines? [duplicate]

File1: Data will be inserted on Thursday Messages has been sent at 6pm Office hours is from 9am to 6pm All the employees are present Output what I want: DMS Data will be inserted on ...
4
votes
5answers
1k views

How to remove all comments from a file?

I have a file with a few lines commented out: foo bar stuff #Do not show this... morestuff evenmorestuff#Or this I just want to print all the uncommented code: foo bar stuff morestuff ...
6
votes
5answers
439 views

reverse order of paragraphs in file

I have a file containing text in paragraphs (lines with text separated by one or more empty lines). I would like to reverse the order of paragraphs (i.e. last paragraph will become the first, ...), ...
0
votes
0answers
19 views

problem in extracting fasta sequence based on header name [on hold]

I'm facing a crazy problem during extracting fasta sequence based on their header name. I have a fasta sequence file and a text file containing their header name. I tried to extract sequence of ...
2
votes
5answers
132 views

How to sort file by character occurrences per line?

I'm quite new to Linux, and I've found quite a bit of useful information on how to do character counts in a file, but is there a way in Linux/terminal to sort a text file by the number of times a ...
2
votes
1answer
65 views

Show nth occurrence of a record

Using awk, how to show the nth occurrence of a record and delete the other occurrence, for example" Input File: 79933,2 79933,4 79933,5 55251,1 49494,4 49494,2 49494,2 49494,1 in this example, ...
1
vote
3answers
44 views

Adding a pattern to each field of the following line

I need some help here please. I have the following input: MX04A; DMX04A; DMX04A; LMX04A; LMX04A; -17.2; -15.3; -14.3; -13.6; -16.8; -15.4; -16.0; -15.3; LH36A; DLH36A; DLH36A; -11; -117.2; -11; ...
1
vote
5answers
321 views

Check number of words in a file that contain a specific letter

Bash command to check number of words in a file that contain letter “a”
2
votes
3answers
488 views

Get last set of non-empty lines

I'm running a cron job which should get only the last result of iostat -d 1 2. This requires some parsing: What is the simplest way to get the last set of non-empty lines from standard input to ...
17
votes
11answers
3k views

There must be a better way to replace single newlines only?

I am in the habit of writing one line per sentence because I typically compile things to LaTex, or am writing in some other format where line breaks get ignored. I use a blank line to indicate the ...
1
vote
2answers
29 views

Append mth and nth columns of a file with the columns of another file

I have 2 files having different number of columns and with huge number of rows: bash-3.00$ cat fileA a1,a2,a3,a4,a5,a6,a7,a8,a9 q,w,,,,y,u,, a,z,,,,q,n,, ......................... z,p,,,,w,e,, ...
14
votes
8answers
8k views

How can I print lines from file backwards (without using “tac”)?

I want to print lines from a file backwards without using tac command. Is there any other solution to do such thing with bash?
2
votes
5answers
70 views

bash expanding to same value instead of line by line value in sed command

I have a file containing numbers in it say numbers.txt as follows. 9374541632553 51243747879841 32030098896914 84654557358238 11000656847765 I want to convert it to the following format. ...
2
votes
3answers
137 views

Remove the ^L (aka \f/FF/form-feed/page-break) character

How do I remove the ^L character and the blank lines that come after it on a unix file? I have tried the below and have been able to remove the VT and spaces but am failing to remove the ^L character ...
3
votes
3answers
129 views

If statement string comparison

I'm reading a csv file with 14 values per line. I want to convert the HEX value from 10th [f10] value to a DEC value. In order to do this if the 7th [f7] value must hold EnHr or EnSt or SpJb or Chem ...
93
votes
3answers
193k views

How can I replace a string in a file(s)?

Replacing strings in files based on certain search criteria is a very common task. How can I replace string foo with bar in all files in the current directory? do the same recursively for sub ...
2
votes
3answers
131 views

How to print the before lines of search keyword with out using grep in a gz file?

How to print the lines before a search keyword without using grep in a .gz file? ID:342N000390AAAAAAAA 07/14/15 10:26 (MV90 ) * Register Data Imported * Warning - No ...
2
votes
2answers
48 views

Awk - keep word “example” and “EXAMPLE” with one input

I am attempting to to only keep customers that contain certain words. Here is an example < newname.txt awk 'BEGIN {FS=","} $7~/USA|United States/' > usa.txt The problem I am having is if they ...
4
votes
6answers
173 views

Extracting lines based on conditions

Each line in a comma-separated file has 5 fields. a,b,c,d,e f,g,c,i, j,k,c,m,n o,p,c,r,s t,u,c,w, x,y,z,aa,bb How can I extract the lines which have c in the 3rd field and their 5th field is NOT ...
0
votes
0answers
32 views

Drawing a histogram from a bash command output

I have the following output from a miRNA Predictor: Read Sequence:hsa-miR-576-3p MIMAT0004796 (22 nt) Read Sequence:gi|182396|gb|M74088.1|HUMFAPAPC Human APC gene mRNA, complete cds (8972 nt) ...
3
votes
3answers
206 views

awk comparison using arrays

I have the following file: 6180,6180,0,1,,1,0,1,1,0,0,0,0,0,0,0,0,4326,4326,,0.440000, 6553,6553,0,1,,1,0,1,1,0,0,0,0,1,0,1,0,4326,4326,,9.000000, ...
2
votes
3answers
233 views

How to 'drop'/delete characters from in front of a string?

I have a string that I would like to manipulate. The string is H08W2345678 how would I be able to manipulate it so the output is just W2345678? Similarly if the I wanted to drop the last 4 ...
3
votes
2answers
771 views

Remove slashes/parent paths from filenames inside CSS and Javascript content

I need to go through all my css and js files and if there is a filename referenced that has any slashes (/) at all then the slash should be removed. What I want is: if any files referenced are ...
3
votes
2answers
637 views

Parsing through log file and printing out regex backreferences

I want to parse my /var/log/maillog to print out the email addresses in from=<> and to=<> (mainly to do a quick check for false positives in the DNS RBLs that I've configured) A maillog ...
2
votes
1answer
2k views

Enable line breaks when assigning multi line outputs to var in bash

I'm using the following very basic shell script I copied off the net to list the contents of my database backup folder and mail them regularly: MYSQLLIST=$(ls -lhG /var/backups/mysql/daily/) ...
3
votes
2answers
831 views

How can I count lines with title-case words of certain length?

How can I create a shell script in unix/linux to find the number of lines in a text file that have words which are 5 to 10 character long and begin with a capital letter?
8
votes
5answers
619 views

How intelligent are unix utilities?

Unix utilities like sort, find, grep, diff et al are very handy to perform quick tasks, sometimes without writing any code at all. I wanted to know what algorithms do they internally use and how do ...
2
votes
1answer
125 views

The cursor to a match, edit, the cursor to the next match, edit — like in a directory?

Algorithmic blackout: problem: find the files that match problem: stack/buffer the matches move the cursor to the next file of the stack and thereby to the location of the match repeat step 2 ...
3
votes
6answers
363 views

Grep the text which matches with a pattern

I was looking for a command which will print the selected portion of a line which matches the conditions. Eg, my text line is like this: p1=X||p2=Y||p3=X||p4=X||p5=X||p6=Y||p7=X I want to print ...
2
votes
2answers
18 views

How to replace value for a given condition in specific column of file

I have file separetd by pipe | and I want that when column 6 contain letter I print 0758000 in column 7 and when the column 6 contain letter A print 0800000 in column 7, I can not find how to do it!!! ...
1
vote
2answers
35 views

Escaping double quotes in string

I have a string message which contains data in double quotes. I need to put \ in front of every double quote how can u do it in shell ? I used Perl and it works fine, but I'm looking for sh command: ...
3
votes
4answers
98 views

Efficient data extraction from multiple files to a single CSV file

I have a large collection of XML files with the same exact structure: $ cat file_<ID>.xml ... ... ... <double>1.2342</double> <double>2.3456</double> ...
1
vote
0answers
44 views

Convert / import flat file with multiple records, one variable per line

I am searching for an efficient way using (UNIX) command line tools to read a flat file into a(ny) database, perhaps after first converting to an intermediate structured format, for example XML or ...
5
votes
4answers
135 views

Remove lines from file up to a pattern, unless the pattern doesn't exist

I am attempting to write a filter using something like sed or awk to do the following: If a given pattern does not exist in the input, copy the entire input to the output If the pattern exists in ...
2
votes
2answers
29 views

Shell script - text processing help

I want to create a ldif modify file to update the users on LDAP. I have an input file(alluserdns.ldif) with list of all Dn entries in ldap: dn: cn=testuser2@abc.com, ou=persons,ou=pp,dc=abc,dc=com ...
3
votes
2answers
343 views

What is the difference between > and >> (especially as it relates to use with the cat program)?

Suppose that I have a file called temp.txt. Using the cat program, I would like to add the contents of this file to the end of myfile.txt -- creating myfile.txt if it does not exist and appending to ...
1
vote
1answer
27 views

Loop over lines of file, partition file and classify each partition

In the following I've put up my code for iterating over lines of a file in order to collect data within a segment from the file in following fashion: Illustration for understanding procedure: segL ...
0
votes
1answer
28 views

How to replace the content of specific column with awk? Tab Delimited File

Here's a similar question but I ran into a problem that was not addressed in the question. How to replace the content of specific column with awk? I want to do something similar, however my file is ...
1
vote
1answer
19 views

AWK: where is my error?

I have a file with :-delimited fields. If the 3rd field of a line is greater than 22, that line should be printed to file44. The file looks like this: 1:2:3:4 1:2:3:411 1:2:33:42 1:2:3:44444 ...
2
votes
3answers
60 views

Read a column from a file and append a specific column of another file

I have two files as such: fileA: Name,Site Name,Product UPEI,Product Name,NMS Flag,Product Model, SW FileB: NE_Name SW ABC 4.4 ASD 4.3 ... ... Note that ...
1
vote
4answers
134 views

UNIX to print a content from a .gz file

I have a .gz file with contents like the following: ID:123455 23-07-2015 mvni warning: RTM post warning : no profile data Using the string no profile data, I want to print the ID. I tried with ...
19
votes
15answers
1k views

How to find line with least characters

I am writing a shell script, using any general UNIX commands. I have to retrieve the line that has the least characters (whitespace included). There can be up to around 20 lines. I know I can use ...
2
votes
2answers
332 views

Merge some tab-delimited files

I have 100 files with 57,816 rows each.I want to do an outer join of these files by merging on basis of the common column. I am using R programming for this but this is very slow. fileList <- ...
8
votes
4answers
3k views

How to delete commands in history matching a given string?

I need to delete all commands in my history matching a string. I've tried: $ history | grep searchstring | cut -d" " -f2 | history -d -bash: history: -d: option requires an argument $ history | grep ...
2
votes
5answers
166 views

Perl script to compare two consecutive lines for the first character and ignore the second line if both first characters are same

Can anyone help how to compare two consecutive lines from a text file for the first character and if both the first characters are same ignore/delete the second line and print into new file only the ...
5
votes
4answers
1k views

How can I delete a trailing newline in bash?

I'm looking for something that behaves like Perl's chomp. I'm looking for a command that simply prints its input, minus the last character if it's a newline: $ printf "one\ntwo\n" | ...
0
votes
0answers
16 views

Adapt text lines to fixed length [duplicate]

With the shell command fmt -w 30 myfile it is possible to truncate the myfile lines so that their length is at most 30 characters. The result is however not a justified file: some lines will be ...
9
votes
7answers
6k views

How to insert the content of a file into another file before a pattern (marker)?

File1 contents: line1-file1 "1" line2-file1 "2" line3-file1 "3" line4-file1 "4" File2 contents: line1-file2 "25" line2-file2 "24" Pointer-file2 "23" ...
7
votes
4answers
316 views

How to insert lines with numbers from X to Y into another file after line Z?

I created two files: echo -e "1\n2\n3\n4\n5" > 123.txt echo -e "a\nb\nc\nd\ne" > abc.txt I want to get the file 123.txt with the following contents: 1 2 3 b c d 4 5 So in other words ...