Manipulation of text by programs, scripts, etc.

learn more… | top users | synonyms

2
votes
3answers
64 views

Sorting numbers and respective units in columns

I have a list like this: 1,kg,cat 1000,g,dog 20,g,apple And I want to sort it by weight. 1 kg and 1000 g would be the same, so I'd like to have my rows sorted by weight. My data set is ...
2
votes
3answers
91 views

How can I extract the text between two strings in a log file?

I think I have a very similar question to this one but I see it was closed due to being unclear so I'll create a new question. I've got a log file that contains one-line entries with multiple ...
1
vote
2answers
42 views

Remove paragraph from file

I have a file (file.php) like this: ... Match user foo ChrootDirectory /NAS/foo.info/ ForceCommand internal-sftp AllowTcpForwarding no GatewayPorts no ...
1
vote
1answer
36 views

How to truncate text lines to N characters maximum? [duplicate]

Given a text file, or the output of a command, how can I truncate it so that every line longer than N characters (usually N=80 in a terminal) gets shorten to N characters maximum?
0
votes
1answer
41 views

Search a file for a string in multiple directories

Can someone please help me with a command to search a file for a string in multiple directories? I am searching for the VIP in httpd.conf in multiple httpd instances. I am using: find ./ -name ...
0
votes
4answers
60 views

Copy same specific parts of a file to another file

How can I copy same specific parts of a file to another file? In the first file, I have: user01: Id name nationality user02: Id name nationality . . . I want to copy each piece of information to a ...
4
votes
1answer
21 views

Sort based on field1, field2 but preserving field 1 sort and ignoring nulls/blanks in 2nd sort

After doing a lot of home work and running multiple test scenarios, I decided to get help from experts Here is my scenario Input data after sorting by Column 1 -bash-3.2$ sort -t'|' -k1 -k2 ...
0
votes
0answers
33 views

Compare specific word counts without variables or expr

I am tasked to write bash command to count number of specific words (given in file) in 2 files and write an error message if number of these words in first file is more than 3 times biggen than in the ...
1
vote
3answers
64 views

How to remove a particular character after a specific word

How do I remove the , character after a specific word? For example, I want to delete the , character after the word IBM IBM, Inc.:APP-VM-Core:4.4.2.0:04, the expected results should be IBM ...
1
vote
1answer
31 views

Search for text inside java command output

I have a folder which have multiple Jar files. * dummy.jar * myjar-xx.jar * myjar-yy.jar Using find, I need to find first file starting with "myjar" Now, I want to grep the contents of ...
1
vote
2answers
49 views

filter blast output for e value

I ran a BlastX against the SwissProt database in with an -evalue threshold of 1e-3. The output looks like: >TCONS_00006082 sp|P43298|TMK1_ARATH 57.89 57 23 2 459 292 751 ...
0
votes
1answer
27 views

Using regular expressions to search a word list. Finding 2 letter words instead of 3. Why?

I am studying regular expressions and I am stumped. Here is my process so far.... I downloaded this file. wget http://www.mieliestronk.com/corncob_lowercase.txt mv corncob_lowercase.txt words.txt ...
1
vote
1answer
23 views

find common lines between multiple files

i have 4 files which are like file A >TCONS_00000867 >TCONS_00001442 >TCONS_00001447 >TCONS_00001528 >TCONS_00001529 ...
4
votes
4answers
326 views

How to fill a file with a stream from /dev/urandom with a specified number of lines ?

I am trying to fill a file with a sequence of random 0 and 1s with a user-defined number of lines and number of characters per line. the first step is to get a random stream of 0 and 1s: cat ...
3
votes
6answers
194 views

Remove everything between second and second last occurance of match

I have string which I would like to format. I would like to remove everything between second ; and second last ;. Input cellular ...
0
votes
1answer
16 views

How to get `pdftotext` to output text in a readable encoding?

I converted a PDF file into a txt file using pdftotext. As an example, i have the sentence "This is the first study on the functional relevance of", notice the f in "first", when i process this sentence ...
3
votes
2answers
104 views

Nested 'awk' in a 'while' loop, parse two files line by line and compare column values

I need some help with a combination of awk & while loop. I have two simple files with columns (normal ones are very large), one representing simple intervals for an ID=10(of coding ...
1
vote
4answers
111 views

SED command to format text file

I need to create a SED command to modify an input file with some conditions explained below. I have an input file something like below. List item Rad# ; ID_KEY ; UNIT_ID 1 ; 30000000004 ; 8417920 ...
2
votes
2answers
34 views

Pipe Viewer - Progress monitor performance consequence

I am writing a batch script to sort through gigs and gigs of data. All of the data is text but the script will take a long long time to execute. I would like to give some visual indication that the ...
0
votes
3answers
40 views

Replacing line with its line number

How can I replace the whole line with no matching pattern just with the number of the line? sed -i "${line}p c\ "$PWD" " file.txt
0
votes
2answers
44 views

sed command to replace a string from one file with entire contents of another file [duplicate]

I want to write a sed (or awk) command to replace a string from one file with the entire contents of another file. Note that second file from which I want to get the content has more than one line. I ...
4
votes
2answers
198 views

BASH base conversion from decimal to hex

In Bash, how does one do base conversion from decimal to another base, especially hex. It seems easy to go the other way: $ echo $((16#55)) 85 With a web-search, I found a script that does the ...
5
votes
2answers
193 views

How to count duplicated last columns without removing them?

I have a file that contains 4 columns. I want to compare the last three columns and count how many times they occur without deleting any of the lines. I just want the count to be present in front of ...
2
votes
3answers
71 views

deduplication of lines in a large file

The size of the file is 962,120,335 bytes. HP-UX ******B.11.31 U ia64 ****** unlimited-user license hostname> what /usr/bin/awk /usr/bin/awk: main.c $Date: 2009/02/17 15:25:17 $Revision: ...
2
votes
1answer
35 views

Updating config files within the script that references them?

I have no problem adding config files to a given script, but automatically updating them is another matter. In this instance I'm providing a few variables to a script, and I'd like the script to be ...
1
vote
3answers
51 views

Add parentheses around numbers

I have a file filled with numbers: 1 6 8 9 10 11 I want to enclose each number in parentheses so basically have the final output as: (1) (6) (8) (9) (10) (11) and so on.
5
votes
8answers
787 views

How to put sentences on separate lines on Linux

I have an assignment to put sentences in a text file on separate lines. Something like this almost works: cat file.txt | tr '.' '\n' But I don't want to lose dots, question marks and exclamation ...
4
votes
3answers
123 views

sum columns based on matching fields

I have encountered a problem that exceeds my basic unix knowledge and would really appreciate some help. I have a large file in the following format: 2 1019 0 12 2 1019 3 0 2 1021 0 2 ...
3
votes
1answer
78 views

how to change the replacement of word in the specific line using sed command?

I have to change a word in the 4th line so I used this command: sed -i '4 s/previous_word/new_word/' filename and it worked perfectly. But if I save the line number in a variable and try the same ...
2
votes
2answers
38 views

vim: using a for-loop-iterator-variable in a search pattern

I am trying to use a for loop iterator variable in a vim search pattern to determine for a range of words how man times they occur in a file. What I do so far is: for i in range(1,40) | %s/SiTg//gn | ...
0
votes
3answers
58 views

Removing duplicate entries in text file

I am doing some sorting with uniq -d command. it's not getting me the desired results. My file is like: >TCONS_00000066 174 206 33 >TCONS_00000066 210 281 72 >TCONS_00000066 227 ...
1
vote
2answers
112 views

Replace string in the first column

I have a file which looks like: >TCONS_00000066_1 [3 - 170] >TCONS_00000066_2 [174 - 206] >TCONS_00000066_3 [2 - 223] >TCONS_00000066_4 [210 - 281] ...
2
votes
2answers
59 views

Using the sed command in Unix

how do you take one word from a file and change it to an absolute path? Basically I need to take one word from a unix file and change it to a absolute directory path in that file. The file name is ...
2
votes
1answer
121 views

Transform .csv into 3 columns and a row

Sample Input: id,Product1,Product2,Product3,Product4 1,0.1,0.3,0.8,0.7 2,0.6,0.7,0.5,0.9 I need output as : id,productname,product_val 1,Product1,0.1 1,Product2,0.3 1,Product3,0.8 1,Product4,0.7 ...
0
votes
1answer
39 views

Compare text files with grep

Lets say I have two files, I want to find goodbye which I can do with grep but what i'm struggling with is once I find goodbye matching the next line with the first column of file 1. File1 sss aa ...
0
votes
2answers
19 views

How to use variables in substitution with vim regular expressions

I have a big file that contains among others serveral lines of the following form: USet07-1 USet07-2 USet08-1 USet08-2 . . . USet22-2 . . . I want to remove the hyphen/dash - from these strings in ...
1
vote
3answers
121 views

how to delete line from the XML file

how to delete only the line from xml file ( with sed/awk or perl one liner line ) that start with: <units> and ended with </Networks> as the follwoing <units><unit ip= ...
-1
votes
0answers
48 views

What logic can I use to replace string in a file?

I am writing a bash automation script to modify certain files from time to time.This script will reset values of .xml and other text files whose values are modified during deployment. The filename, ...
1
vote
2answers
35 views

How to only leave in given characters in a file?

Purpose: tr -cd '\12\40-\176' but with accents, ex.: á, í, ű, ő, ö, ü, ű, ú, ó, é. In octal: "á" is 341 "\n" is 12 So I tried: [user@notebook ~]$ printf 'XXXá\nXXX' | tr -cd '\12\341' | ...
2
votes
2answers
45 views

Find out on which line in text file is matching word

Is there any way to find out on which line of the text document is some word which is matching pattern for example with grep or something. Thanks.
0
votes
1answer
77 views

Read only specific lines from text file and save matches into variables

I've got this problem. I need to extract some words from text file, which I find by specific pattern in that text. For example: /this/is/path: file1.txt,date-3 file2.txt,date-2 /this/is/path2: ...
2
votes
3answers
86 views

How to cut away the end of a textfile after a specified line

I have == Changelog == in a textfile and I want to create a new file where this line and everything after it will be deleted. I am sure the answer is here somewhere, I have tried to search, now my ...
1
vote
2answers
50 views

How to read only the second last line of a file

I have a client that updates/writes into myfile.csv arbitrarily. I've written the following code, I left the MySQL snippet out: while true do awk_variables_value_array=`tail -n2 myfile.csv | awk ...
1
vote
1answer
79 views

Bash: Looping through a string

I have a variable Vmac that stores values of multiple MAC addresses, collected from wmic call as follows: Vmac=`wmic --delimiter="," --authentication-file=/path/to/file //IP-ADDR "Select ...
1
vote
2answers
41 views

Some characters disappear after processing a text file

I've a file named addresses.txt : address1@domain.com address2@domain.com address3@domain.com I'd like to echo "address1@domain.com.html" using for loop. Here's what I did : cat addresses.txt | ...
-1
votes
1answer
22 views

Remove line breaks in a FASTA file

I have a fasta file where the sequences are broken up with newlines. I'd like to remove the newlines. Here's an example of my file: >accession1 ATGGCCCATG GGATCCTAGC >accession2 GATATCCATG ...
0
votes
0answers
28 views

match two files [duplicate]

I have 2 files. The first, fileA looks like TCONS_00000066 XLOC_000030 - u q1:XLOC_000030|TCONS_00000066|0|0.000000|0.000000|0.000000|0.000000|- TCONS_00000130 XLOC_000057 - u ...
1
vote
2answers
69 views

match first column of file a with paragraphs of file b

I have 2 files. The first, fileA looks like TCONS_00000066 XLOC_000030 - u q1:XLOC_000030|TCONS_00000066|0|0.000000|0.000000|0.000000|0.000000|- TCONS_00000130 XLOC_000057 - ...
1
vote
3answers
45 views

Remove duplicate entries from a CSV file

I've got a [csv] file with duplicate datum reprinted ie the same data printed twice. I've tried using sort's uniq by sort myfile.csv | uniq -u however there is no change in the myfile.csv, also I've ...
1
vote
1answer
33 views

Replace a string in a text file using a shell script not working properly

I have a text file containing this: 0 0 -1 0 1 0 0 0 0 -1 0 0 1.5 0.0 1.0 1 and I want to replace the 1.0 to 2.0 so that it becomes: 0 0 -1 0 1 0 0 0 0 -1 0 0 1.5 0.0 2.0 1 so I use this ...