Questions tagged [perl]

Perl is a high-level, general-purpose, interpreted, dynamic programming language. It was originally developed by Larry Wall as a general-purpose Unix scripting language to make report processing easier, over a period with support from open source community, it has evolved and matured to be used for graphics programming, system administration, network programming, finance, bioinformatics, and other engineering applications.

724
votes
7answers
1.5m views

How can I replace a string in a file(s)?

Replacing strings in files based on certain search criteria is a very common task. How can I replace string foo with bar in all files in the current directory? do the same recursively for sub ...
38
votes
7answers
5k views

Linux equivalent to PowerShell's “one-to-many” remoting

Coming from Windows administration, I want to dig deeper in Linux (Debian). One of my burning questions I could not answer searching the web (didn't find it) is: how can I achieve the so called "one-...
23
votes
5answers
34k views

How to grep for text in a file and display the paragraph that has the text?

Below is the text in the file: Pseudo name=Apple Code=42B state=fault Pseudo name=Prance Code=43B state=good I need to grep for "42B" and get the output from the above text like: Pseudo name=...
16
votes
4answers
82k views

compare two columns of different files and print if it matches

I am using Solaris 10 and so grep options involving -f don't work. I have two pipe-separated files: file1: abc|123|BNY|apple| cab|234|cyx|orange| def|kumar|pki|bird| file 2: abc|123| kumar|pki| ...
22
votes
1answer
1k views

What's with all the renames: prename, rename, file-rename?

On my Debian system (well, my LMDE system, but close enough), I have at least 3 different rename programs: /usr/local/bin/rename : This is a Perl script, written by Tom Christiansen. Oddly enough, I ...
30
votes
10answers
54k views

How to insert the content of a file into another file before a pattern (marker)?

File1 contents: line1-file1 "1" line2-file1 "2" line3-file1 "3" line4-file1 "4" File2 contents: line1-file2 "25" line2-file2 "24" Pointer-file2 "23" line4-...
42
votes
3answers
32k views

What does ${1+“$@”} mean in a shell script, and how does it differ from “$@”?

In the Perl documentation, perlrun(1) suggests launching Perl scripts using a bilingual shell/Perl header: #!/bin/sh #! -*-perl-*- eval 'exec perl -x -wS $0 ${1+"$@"}' if 0; What does ${1+"$@"} ...
14
votes
7answers
42k views

Repeat each line multiple times

Would like to have each line in a file repeated a fixed number of times. e.g. have each line repeated four times: a b c becomes: a a a a b b b b c c c c I've done some searching, and there are a ...
26
votes
2answers
2k views

Security implications of running perl -ne '…' *

Apparently, running: perl -n -e 'some perl code' * Or find . ... -exec perl -n -e '...' {} + (same with -p instead of -n) Or perl -e 'some code using <>' * often found in one-liners ...
5
votes
5answers
4k views

Replace strings in a file based on a list of strings and a list of corresponding replacements

I am trying to replace strings in a file A: Hello Peter, how is your dad? where is mom? where the strings to be replaced are in file B: Peter dad mom and their corresponding replacements are in ...
7
votes
1answer
1k views

How can I extract/change lines in a text file whose data are separated into fields?

How can I manipulate field-based data from the commandline? For example How can I print only lines whose Nth field is foo? How can I print only lines whose Nth field isn't foo? How can I print only ...
19
votes
1answer
19k views

Substitute pattern within a file with the content of other file

I have a text file (devel.xml). I added the word REPLACETHIS to it in order to replace this string with the content within a different file (temp.txt). The closest thing I have is this: sed -i -e "/...
8
votes
7answers
7k views

What is the easiest way to add a string on the beginning of every line of the file from the command line?

I am looking for a way to add some string to the beginning of every line (same string for every line). Not something customizable but rather something that will be easy to remember and available on ...
7
votes
4answers
7k views

Average rows with same first column

Given a file with two columns: Id ht 510 69 510 67 510 65 510 62 510 59 601 29 601 26 601 21 601 20 I need a way to coalesce all rows with the same ID into one that has an average height. In this ...
0
votes
1answer
2k views

/bin/sh: ./check-dependencies.pl: not found — but check-dependencies.pl exists!

I'm trying to build Moses, but when I run make I get the following output: minakshi@minakshi-Vostro-3500:~/Desktop/working-dir/moses/scripts$ make release # Compile the parts make all make[1]: ...
34
votes
9answers
22k views

How can you combine all lines that end with a backslash character?

Using a common command line tool like sed or awk, is it possible to join all lines that end with a given character, like a backslash? For example, given the file: foo bar \ bash \ baz dude \ happy ...
5
votes
4answers
2k views

concatenate lines based on first char of next line

I am looking for away contact lines based on the next line. So far the only way I see is to create a shell script that will read line by line and will do something along these lines: while read line ...
3
votes
3answers
422 views

Four random words from a list with Perl!

For start, we know how can we output random lines from a txt file: perl -MList::Util -e 'print List::Util::shuffle <>' words.txt But we need a general solution (perl is usually installed on ...
6
votes
2answers
981 views

How exactly does perl's -0 option work?

According to man perlrun: -0[octal/hexadecimal] specifies the input record separator ($/) as an octal or hexadecimal number. If there are no digits, the null character is the separator....
5
votes
1answer
2k views

Renaming files to have lower case extensions with 'rename'

I'm trying to currently rename a large set of files and have been using quite kludgy methods to do so, such as: rename 's:(.*)\.MOV:$1.mov:g' *.MOV rename 's:(.*)\.JPG:$1.jpg:g' *.JPG What I'd ...
10
votes
6answers
22k views

How to replace only the Nth occurrence of a pattern in a file?

How to replace the third occurrence of the string in the file using sed command. Example: Change only the third occurrence of is to us in the file. My input file contains: hai this is linux. hai ...
5
votes
4answers
8k views

How to filter IPv4 and IPv6 addresses?

echo "1.1.1.1" | awk '/[0-9]*\.[0-9]*\.[0-9]*\.[0-9]*/ {print $1}' How can I filter IPv4 addresses in a script, and how can I filter IPv6 addresses. I mean more precisely then this so the oneliner ...
1
vote
1answer
107 views

Remove a specific latex command from the text AND closing bracket behind it

How to remove a specific latex command from the text AND closing bracket behind it, but to keep the text inside the brackets? The command in the following example to remove is \edit{<some staff>}...
6
votes
1answer
57k views

convert CSV to XLS file on linux [closed]

The following Perl script can convert CSV file to XLS file the problem is that I need to install on customer Linux machine many Perl modules in order to run this Perl script , and actually I cant ...
13
votes
4answers
28k views

How to open terminal, split to 9 terminals and switch between them using one script?

I would like to open a terminal, split it to lets say 9 parts (3x3) and execute some bash script. But for each terminal part different script. Can this be done using perl, python or even bash? How ...
14
votes
5answers
8k views

Remove duplicate lines while keeping the order of the lines

[root@server]# awk '!seen[$0]++' out.txt > cleaned awk: (FILENAME=out.txt FNR=8547098) fatal error: internal error Aborted [root@server]# The ""server"" has: 8 GByte RAM + 16 GByte SWAP, x>300 ...
11
votes
2answers
18k views

Replacing missing value blank space with zero

I have input.txt tab-delimited text file around 30K lines, I would like to check each row (s1..s30K lines) for missing value (i.e blank space) and fill the missing value with zero value.See out.txt ...
19
votes
2answers
36k views

How to know reason of a process going to sleep state and wake it up?

When I do 'top' , I see my perl script process in 'S' state. Is there a way to know, what is making this perl process go in sleep state and ways to wake it up? Any way to debug to give more insights ...
12
votes
2answers
2k views

Using empty line as context “group-separator” for grep

I need grep output with context, in color, and blank lines as group separator. In this question, I learned how to define custom group-separator, and I have constructed my grep command like this: grep ...
10
votes
6answers
4k views

Is there any alternative to grep's -A -B -C switches (to print few lines before and after )?

grep -A 2 -B 3 prints 2 lines after the grep string and prints 3 lines before. grep -C 3 prints 3 Lines before and 3 lines after Unfortunately, the grep I'm using does not support these options. ...
8
votes
2answers
9k views

Replace string with sequential index

Can someone suggest an elegant way to accomplish this? Input: test instant () test instant () ... test instant () //total 1000 lines output should be: test instant1 () test ...
6
votes
2answers
9k views

How to print incremental count of occurrences of unique values in column 1

I'm trying to come up with an solution to this problem, I need to incrementally count and then print the counts of the unique values in column 1 of a tab delimited text file. Here is an example: ...
11
votes
3answers
880 views

Is there a way to make perl -i not clobber symlinks?

A friend of mine points out that if you do: perl -pi.bak -e 's/foo/bar/' somefile when "somefile" is actually a symlink, perl does just what the docs say it will do: It does this by renaming the ...
10
votes
2answers
803 views

How is the inplace modification of a file being done?

What does the "inplace" modification of a file e.g. via sed -i or perl -i mean? My question is about how this inplace modification is done. Is the file copied the modification is done in the copy and ...
6
votes
7answers
28k views

extract the 4th column from a csv file using unix command

I have a csv file formatted as below. "col1","col2","col3","col4" "1","text1","<p>big html text</p> ","4th column" "2","text2","<p>big2 html2 text2</p> ",...
2
votes
5answers
318 views

Can the string “abcedf” be matched to string “bafcde” in a single line command?

I am planning to implement an indexing structure in my program. For example, if I have 100 rows in the table, I will number these rows from 1 to 100 in another column by appending an _ to the end of ...
0
votes
4answers
1k views

How to sum up values of each two rows across their line in linux

I have a data like this: input.txt 1 0000100101000000 1 0000010100000000 2 1110000001000000 2 1111000000001000 3 0000000111111111 3 1111111100000000 4 8888345500000000 4 0000000000000000 and I want ...
6
votes
5answers
861 views

Replace a string before a certain line

I have one file like: ID_SOUR_CALENDAR BIGINT NOT NULL DEFAULT 0 COMPRESS 0 , UNIQUE PRIMARY INDEX ( CALENDAR_DATE ); ID , ID_SOUR , PRIMARY INDEX ( CALENDAR_DATE ); I want to replace the ',' by ')'...
5
votes
2answers
460 views

Could someone explain this shebang line which uses sh and then does exec perl?

This is from Programming Perl, Fourth Edition. It's about executing a Perl script. Finally, if you are unfortunate enough to be on an ancient Unix system that doesn’t support the magic #! line, ...
4
votes
2answers
3k views

Replace lines matching a pattern with lines from another file in order

I want to replace the lines matching a pattern from one file from the lines in order from another file, for example, given: file1.txt: aaaaaa bbbbbb !! 1234 !! 4567 ccccc ddddd !! 1111 we like to ...
4
votes
2answers
1k views

How to write this pseudocode where condition

This question is extension of this question about listing questions of directories. I get a problem that I need to understand the original section of the text. Current output where you see the ...
4
votes
3answers
4k views

replace column of a csv file with column from another file

I have two sample files like this: $ cat file1 abc,sachin cat,kumar $ cat file2 xyz pressure $ cat file3 xyz,sachin pressure, kumar I want the first column of file1 to be replaced with file2. ...
4
votes
5answers
741 views

Count unique associated values in awk (or perl)

I've already found "How to print incremental count of occurrences of unique values in column 1", which is similar to my question, but the answer isn't sufficient for my purposes. First let me just ...
3
votes
3answers
344 views

sed regular expression behaving differently than in vim and perl?

Here's a sample 'test.txt' abcdX1yad45das abcdX2fad45das abcdX3had45das abcdX4wad45das abcdX5mad45das sample desired output: X1yad X2fad X3had X4wad X5mad I could get it working in vim with: :% s/...
2
votes
4answers
2k views

Merge lines between keywords into one-line comma separated values

Between first occurrence of Cat to next occurrence of Cat, it should create a separate line with delimiter as ",". File input as below. Cat AA BB CC Cat AA-1 BB-1 CC-1 Output expected: Cat,AA,BB,...
11
votes
5answers
17k views

How can I convert timestamps in a column to a date?

I have a file containing this: 1415602803,LOGIN SUCCESS,AUTH,user2,192.168.203.63,10.146.124.73,59996,22 1415602807,LOGIN SUCCESS,AUTH,user1,172.24.31.10,172.32.1.1,48191,22 1415602811,LOGIN FAILED,...
5
votes
3answers
4k views

How to remove all white spaces just between brackets [] using Unix tools? [duplicate]

Replace text between brackets Input testing on Linux [Remove white space] testing on Linux Output testing on Linux [Removewhitespace] testing on Linux So, how can we just remove all the white ...
4
votes
4answers
112 views

Replace pattern each time with a different string (taken from external file)

I have an input: a b TOCHANGE c d e TOCHANGE where I need to change the patterns "TOCHANGE" using an external file : line1 line2 ... so that I get the following output : a b line1 c d e ...
3
votes
2answers
9k views

redirect output of command to /dev/null

I run the following line from my bash script in order to run the script /usr/run.pl on remote machines xargs -P "$numprocs" -I '{}' -n 1 -- perl -e 'alarm shift; exec @ARGV' -- "$timeout" ssh -...
2
votes
4answers
1k views

How to calculate the sum of the data that have the same ID in the first column?

I want to sum the values of the rows of my data that have the same ID in the first column. My data looks like data.txt Id a b c d e 1 1 2 3 4 5 1 2 3 4 5 ...