Manipulation of text by programs, scripts, etc.

learn more… | top users | synonyms

4
votes
6answers
228 views

Turn list into single line with delimiter

I have to take a list (loads) of IP addresses in this format: 134.27.128.0 111.245.48.0 109.21.244.0 and turn them into this format with a pipe in-between (IPs made up) 134.27.128.0 | ...
11
votes
7answers
9k views

Text processing - join every two lines with commas

I have more than 1000 lines in a file. The file starts as follows (line numbers added): 1. Station Name 2. Station Code 3. A N DEV NAGAR 4. ACND 5. ABHAIPUR 6. AHA 7. ABOHAR 8. ABS 9. ABU ROAD 10. ...
1
vote
1answer
32 views

Trouble with Sed Syntax

I'm modifying raw PCL/PS files (mixed) and for some reason I can't get my Sed syntax right for the true beginning that I want it to grab. Here's a sample output from the strings command: *c50B *c0P ...
2
votes
6answers
122 views

Join lines of text with repeated beginning

I have a long text file (a tab-file for stardict-editor) which consists of lines in the following format: word1 some text word1 some other text word2 more text word3 even more and would like to ...
51
votes
2answers
92k views

How can I replace a string in a file(s)?

Replacing strings in files based on certain search criteria is a very common task. How can I replace string foo with bar in all files in the current directory? do the same recursively for sub ...
-1
votes
2answers
23 views

Divide value in 1st file by value in 2nd file [on hold]

I have a file with a value and a second file with the value 1024. Now I want to to divide the value in the first file with the value in the second file and store the result back into the first file. ...
0
votes
2answers
30 views

Select lines and format

Is there a way to format the following output so that only lines 1 and the respective lines that state the "rule" name print out? Depending on the configuation, the "rule" line could vary. It could be ...
0
votes
3answers
38 views

How to find the word after a given pattern using cut?

I have a file that looks like this: -config1 -config2 param2 -config3 param3a param3b -config4 param4 and I can use the following to extract param2: echo $(<file) | cut -d' ' -f3 However I ...
1
vote
2answers
35 views

SED replace after ignoring

There are many lines in a text file and I would like to change the value true to false using a one line script. bash-3.2$ grep online test.txt # online = false --> ONE SHOT import (delete all ...
0
votes
1answer
38 views

Modify part of line if keyword is found using sed? [duplicate]

I want to modify a line if a specific keyword is found in a line. CS_CODE, SM_NUM, PORT_NUM, DEV_PORT_NUM, DIRNUM, MAIN_DIRNUM, BILL_NO, ADR_NAME In above line, if 'PORT_NUM' is present then ...
5
votes
6answers
4k views

How to insert the content of a file into another file before a pattern (marker)?

File1 contents: line1-file1 "1" line2-file1 "2" line3-file1 "3" line4-file1 "4" File2 contents: line1-file2 "25" line2-file2 "24" Pointer-file2 "23" ...
2
votes
1answer
49 views

How to select lines and reformat command output?

Is there a way to format the following output so that only lines 1 and 4 print out? It would be best if the first line and fourth line could come out on the same line like this: sw pool test(no ...
1
vote
5answers
52 views

keep out line break

I have some files with this structure: 2015-03-25 17:08:17 sysUpTimeInstance 93474;^M 1.ValueforState=2500 I want to replace the line break, and leave the third line with the second line, I mean, ...
0
votes
1answer
34 views

How to match parameter and replace the value after “=” separator

the perl one liner command to replace Value in file is simple: perl -i 's/oldval/newval/g' file.txt but what should be the perl one liner syntax in case we want to replace the value on specific ...
-2
votes
1answer
38 views

how to print new word after two lines using awk

I need to insert a blank line after two lines in a file that i did with cat rat hat pot NOT lot big big big sit sit sit awk '{print};(NR%2==0){print ""}' cat rat hat pot NOT ...
1
vote
2answers
53 views

sort only the first column and uniq

I have a list as such below: 1,cat 1,dog 2,apple 3,human I'd like an output like this: 1,cat,dog 2,apple 3,human So value 1 from column 1 contains the value of cat and dog from ...
2
votes
3answers
94 views

Sorting numbers and respective units in columns

I have a list like this: 1,kg,cat 1000,g,dog 20,g,apple And I want to sort it by weight. 1 kg and 1000 g would be the same, so I'd like to have my rows sorted by weight. My data set is ...
1
vote
2answers
116 views

Replace string in the first column

I have a file which looks like: >TCONS_00000066_1 [3 - 170] >TCONS_00000066_2 [174 - 206] >TCONS_00000066_3 [2 - 223] >TCONS_00000066_4 [210 - 281] ...
6
votes
7answers
672 views

Removing a specific line from a file

Somewhere in the middle of my CSV file is this line: Products below this line are out of stockNumber, month, year, reference, store Note:Number, month, year, reference and store are the CSV fields. ...
2
votes
3answers
106 views

How can I extract the text between two strings in a log file?

I think I have a very similar question to this one but I see it was closed due to being unclear so I'll create a new question. I've got a log file that contains one-line entries with multiple ...
1
vote
2answers
49 views

Remove paragraph from file

I have a file (file.php) like this: ... Match user foo ChrootDirectory /NAS/foo.info/ ForceCommand internal-sftp AllowTcpForwarding no GatewayPorts no ...
0
votes
1answer
41 views

Search a file for a string in multiple directories

Can someone please help me with a command to search a file for a string in multiple directories? I am searching for the VIP in httpd.conf in multiple httpd instances. I am using: find ./ -name ...
0
votes
4answers
62 views

Copy same specific parts of a file to another file

How can I copy same specific parts of a file to another file? In the first file, I have: user01: Id name nationality user02: Id name nationality . . . I want to copy each piece of information to a ...
1
vote
1answer
36 views

How to truncate text lines to N characters maximum? [duplicate]

Given a text file, or the output of a command, how can I truncate it so that every line longer than N characters (usually N=80 in a terminal) gets shorten to N characters maximum?
4
votes
1answer
21 views

Sort based on field1, field2 but preserving field 1 sort and ignoring nulls/blanks in 2nd sort

After doing a lot of home work and running multiple test scenarios, I decided to get help from experts Here is my scenario Input data after sorting by Column 1 -bash-3.2$ sort -t'|' -k1 -k2 ...
0
votes
0answers
33 views

Compare specific word counts without variables or expr

I am tasked to write bash command to count number of specific words (given in file) in 2 files and write an error message if number of these words in first file is more than 3 times biggen than in the ...
1
vote
3answers
71 views

How to remove a particular character after a specific word

How do I remove the , character after a specific word? For example, I want to delete the , character after the word IBM IBM, Inc.:APP-VM-Core:4.4.2.0:04, the expected results should be IBM ...
1
vote
1answer
34 views

Search for text inside java command output

I have a folder which have multiple Jar files. * dummy.jar * myjar-xx.jar * myjar-yy.jar Using find, I need to find first file starting with "myjar" Now, I want to grep the contents of ...
1
vote
2answers
51 views

filter blast output for e value

I ran a BlastX against the SwissProt database in with an -evalue threshold of 1e-3. The output looks like: >TCONS_00006082 sp|P43298|TMK1_ARATH 57.89 57 23 2 459 292 751 ...
0
votes
1answer
28 views

Using regular expressions to search a word list. Finding 2 letter words instead of 3. Why?

I am studying regular expressions and I am stumped. Here is my process so far.... I downloaded this file. wget http://www.mieliestronk.com/corncob_lowercase.txt mv corncob_lowercase.txt words.txt ...
4
votes
4answers
331 views

How to fill a file with a stream from /dev/urandom with a specified number of lines ?

I am trying to fill a file with a sequence of random 0 and 1s with a user-defined number of lines and number of characters per line. the first step is to get a random stream of 0 and 1s: cat ...
7
votes
10answers
4k views

Decoding URL encoding (percent encoding)

I want to decode URL encoding, is there any built-in tool for doing this or could anyone provide me with a sed code that will do this? I did search a bit through unix.stackexchange.com and on the ...
5
votes
2answers
2k views

tr complains of “Illegal byte sequence”

I'm brand new to UNIX and I am using Kirk McElhearn's "The Mac OS X Command Line" to teach myself some commands. I am attempting to use tr and grep so that I can search for text strings in a regular ...
3
votes
1answer
331 views

Strip most frequent words from text

I have a simple problem, but unfortunately I don't know where to even start (I'm just starting out). So, what I want to do is ultimately increase my vocabulary. I got the idea to strip the most ...
3
votes
2answers
115 views

Nested 'awk' in a 'while' loop, parse two files line by line and compare column values

I need some help with a combination of awk & while loop. I have two simple files with columns (normal ones are very large), one representing simple intervals for an ID=10(of coding ...
24
votes
10answers
7k views

A standard tool to convert a byte-count into human KiB MiB etc; like du, ls1

Is there a standard tool which converts an integer count of Bytes into a human-readable count of the largest possible unit-size, while keeping the numeric value between 1.00 and 1023.99 ? I have my ...
1
vote
1answer
25 views

find common lines between multiple files

i have 4 files which are like file A >TCONS_00000867 >TCONS_00001442 >TCONS_00001447 >TCONS_00001528 >TCONS_00001529 ...
3
votes
6answers
196 views

Remove everything between second and second last occurance of match

I have string which I would like to format. I would like to remove everything between second ; and second last ;. Input cellular ...
1
vote
3answers
51 views

Add parentheses around numbers

I have a file filled with numbers: 1 6 8 9 10 11 I want to enclose each number in parentheses so basically have the final output as: (1) (6) (8) (9) (10) (11) and so on.
3
votes
3answers
12k views

Loop over a file and read values from two columns into variables

I have a file (data.dat) which contains 3 columns. I want to read in two of them with the shell script and fill the contents into a template file. The file looks like this: 12 180 2390 14 177 2210 16 ...
0
votes
4answers
872 views

How to get data count between two strings with multiple occurences in the same file

I have the data in my file as START-OF-FIELDS ID NAME DEPT END-OF-FIELDS START-OF-DATA 1|joy|cs 2|sam|ec END-OF-DATA START-OF-FIELDS ID NAME DOB DEPT ADDRESS END-OF-FIELDS START-OF-DATA ...
11
votes
3answers
577 views

/usr/bin/ptx: Can you provide a use case or two?

I was going through the list of files included in coreutils and I was able to come up with an example of how I could personally use all of the commands provided except for ptx. Can you give one or two ...
2
votes
2answers
36 views

Pipe Viewer - Progress monitor performance consequence

I am writing a batch script to sort through gigs and gigs of data. All of the data is text but the script will take a long long time to execute. I would like to give some visual indication that the ...
0
votes
1answer
18 views

How to get `pdftotext` to output text in a readable encoding?

I converted a PDF file into a txt file using pdftotext. As an example, i have the sentence "This is the first study on the functional relevance of", notice the f in "first", when i process this sentence ...
1
vote
4answers
113 views

SED command to format text file

I need to create a SED command to modify an input file with some conditions explained below. I have an input file something like below. List item Rad# ; ID_KEY ; UNIT_ID 1 ; 30000000004 ; 8417920 ...
13
votes
2answers
52k views

Return only the portion of a line after a matching pattern

So pulling open a file with cat and then using grep to get matching lines only gets me so far when I am working with the particular log set that I am dealing with. It need a way to match lines to a ...
1
vote
6answers
240 views

Replace \n with blank in a file

I want to replace \n with blank Input file1 A\n D\n Output file2 I want A D I tried below commands but not able to replace please do needful. cat OUT | tr '\n' ' '
0
votes
3answers
40 views

Replacing line with its line number

How can I replace the whole line with no matching pattern just with the number of the line? sed -i "${line}p c\ "$PWD" " file.txt
2
votes
3answers
73 views

deduplication of lines in a large file

The size of the file is 962,120,335 bytes. HP-UX ******B.11.31 U ia64 ****** unlimited-user license hostname> what /usr/bin/awk /usr/bin/awk: main.c $Date: 2009/02/17 15:25:17 $Revision: ...
5
votes
8answers
789 views

How to put sentences on separate lines on Linux

I have an assignment to put sentences in a text file on separate lines. Something like this almost works: cat file.txt | tr '.' '\n' But I don't want to lose dots, question marks and exclamation ...