Manipulation of text by programs, scripts, etc.

learn more… | top users | synonyms

2
votes
2answers
73 views

sed: substitute text with a string that contains special characters

I have a lot of text file with specific tag called "Comment", example I have a file called 3dchess.sh, I need change the tag: Comment=3D chess for X11 to Comment='<span ...
4
votes
2answers
153 views

Replace string with sequential index

Can someone suggest an elegant way to accomplish this? Input: test instant () test instant () ... test instant () //total 1000 lines output should be: test instant1 () test ...
0
votes
3answers
65 views

removing spaces from first column

My input file has positions in first column with different number of spaces (or no space) 16504 16516 1650811 16520 1651 16524 16516111 16528 165204 16532 ...
1
vote
5answers
67 views

Replicate a column in a one-line CSV file

I have a file that consists of a single line: a,x,b,c,d,e I want to convert this into a,x,b,x,c,x,d,x,e,x Which is the easiest way to achieve this? A solution based on Python will be most ...
1
vote
1answer
17 views

Wierd output using paste with serial option

I have a testA.txt with contents as shown below [jiewmeng@JM textFiles]$ cat testA.txt The quick brown fox jumped over the lazy dog. paste normally works [jiewmeng@JM textFiles]$ paste testA.txt ...
2
votes
3answers
57 views

How to optimize this oneliner? (if second column empty, put the line in another file)

I have a little "oneliner": while read ONELINE; do if [[ -n "`echo $ONELINE | awk '{print $2}'`" ]]; then echo "$ONELINE" >> LIST-ok.txt else echo "$ONELINE" >> ...
1
vote
3answers
52 views

parsing file from ls -laLR

ls -laLR /home/tools > all_site1.txt I need the state of all folders and files in subfolders of /home/tools. With this command I put all in this file. What I need now is a solution how to parse ...
5
votes
4answers
42 views

Increment numbers inside strings by an offset

I want to search a text file for strings like these. [task0] [task1] ... [task1613] and increment each by a given offset, in this case by 3640. So, with the example I gave above, I'd get the ...
0
votes
1answer
27 views

is it possible to reverse the output of “filters” command?

On Linux, the filters package could be great fun, really. Is there any way to reverse the output to what it basically was? echo "Hi. My name is Chirp Chirp" | ky00te Hy. My name ys Chyrrp Chyrrp Is ...
3
votes
2answers
82 views

paste command: setting (multiple) delimiters

In Linux, I have the following problem with paste from (GNU coreutils) 8.13: Trying to set another delimiter than the default (TAB) results in either just printing the first character of the defined ...
5
votes
4answers
210 views

reverse order of paragraphs in file

I have a file containing text in paragraphs (lines with text separated by one or more empty lines). I would like to reverse the order of paragraphs (i.e. last paragraph will become the first, ...), ...
5
votes
2answers
45 views

Sort one column only, keeping remaining columns untouched

How to (numerically) sort a specific column in a text file, without affecting other columns (irrespective of whether they are sorted or unsorted)? On other threads I found -s argument, but it does ...
4
votes
4answers
151 views

Print line after nth occurrence of a match

I am looking to display the line 4598 in the following file. Effectively I want to display the line AFTER the nth occurrence of a match. In this case, the line after the 3rd occurrence of <Car>. ...
4
votes
5answers
80 views

Reformat each two-line sequence into two columns

I would like to tab my printout: 1 2 3 4 5 6 7 8 to 1 2 5 6 3 4 7 8 How to adjust it using simple command line (awk or sed)??
1
vote
4answers
66 views

Count rows with specific integer in a column

I have 6 columns, each with multiple rows. I want to count the number of rows which have the integer 4 or 5 in the fifth column. A1 jhfj jdhfjkhd kdkfjjh 5 jhsdjkfh A2 ujhf jhdfhsd dsfkks 4 ...
0
votes
4answers
53 views

What are alternative tools for searching files in full content?

I realized that most of the text related tools I use, grep/egrep/awk/sed are line-oriented. I guess that is a simplified approach to search in files but I was wondering if there is another set of ...
1
vote
1answer
57 views

Insert a line after a block of text with sed

I have text file: file.conf ### Option: LogFile # Name of log file. # If not set, syslog is used. # # Mandatory: no # Default: # LogFile= ### Option: LogFileSize # Maximum size of ...
5
votes
1answer
98 views

counting multiple patterns in a single pass with grep?

I've written a grep loop to iteratively count DNA trinucleotides within a gzipped DNA fasta file containing DNA sequences e.g. declare -a tri=(AAA AAC AAG AAT CAA .. etc) for i in ${tri[@]} do ...
-1
votes
1answer
49 views

Importing an excerpt from a file into a script

I'm trying to import an excerpt from a file into a script for processing. LIST=$(sed '1,/+++NETWORKLIST+++/d' < /path/to/file | sort -t ";" -k2,2r) My problem is that this script is not deleting ...
2
votes
3answers
124 views

Print Column if values is between two other values

I am fairly new to data analysis and I have a problem with filtering some of my values. I have data arranged in a file with four columns like this: A 1 10 5 B 10 100 120 C ...
2
votes
4answers
115 views

Create an ASCII art table from tabular data

Given perhaps comma- or tab-delimited input, I'd like to present a series of appropriately padded columns to stdout, so I can easily scan columnar information which would otherwise present rather ...
3
votes
4answers
81 views

Replace all newlines to space except the last

How can I replace all newlines with space except the last newline. I can replace all newline to space using tr but how I can do it with some exceptions?
2
votes
2answers
46 views

print all matches or replace all strings in a BIG file which is NOT line organised (no line separators)

The 5GB files I have are streams of data rows formed: {datarow1...},{datarow2...},...,{datarowN...} so actually could say that there are lines {}, and even line separators, but coming as a ...
1
vote
1answer
258 views

grep to extract a substring from a huge string

I am extracting certain information from a website using curl command. I need to extract a certain subsection of the string using grep command. For example, using curl command I am getting the HTML ...
2
votes
2answers
47 views

Replacing a pattern with another between first and second occurrence of a pattern in file

Want the comma's (,) replaced by the alphabet u between the first occurrence of y till the second occurrence of y. Given below is the sample input file. 100y00893225900,008932259,06,2013,NOKIA Q ...
18
votes
3answers
989 views

Unix command to check if any two lines in a file are same?

Is there a unix command that can check if any two lines in a file are the same? For e.g. Consider a file sentences.txt This is sentence X This is sentence Y This is sentence Z This is sentence X ...
-2
votes
3answers
71 views

Deleting Lines, which match a particular Identifier from another file

I have 2 files. File 1, has an identifier (eg. D7MHBF:11:1449:1988) and every new entry starts with @. It has few more fields, which are not important in our analysis. File 2 consists of a column of ...
18
votes
2answers
517 views

Is there a reason why ls does not have a --zero or -0 option

This question was prompted by questions about ls' -1 option and the recurring tendency of people to ask question and answers that includes processing the output of ls. This reuse of the output ls ...
6
votes
1answer
166 views

How can I replace a string in a file(s)?

Replacing strings in files based on certain search criteria is a very common task. How can I replace string foo with bar in all files in the current directory? do the same recursively for sub ...
2
votes
1answer
32 views

Count and merge consecutive patterns

I'm searching for a short snippet to find, count and merge consecutive duplicates using standard tools or a common scripting language. Say our input is: 1 2 2 2 7 22 a b b c c c c c d dd 2 2 c c ...
3
votes
7answers
106 views

How to combine contents of two files having the same value in a column?

File1: 00:00 274 00:04 476 00:05 450 00:06 499 00:07 373 00:08 206 00:09 471 00:10 154 File2: 00:00 183 00:01 60 00:02 344 00:03 540 00:04 450 00:07 348 00:09 473 00:10 203 Desired output: ...
0
votes
2answers
66 views

Using AWK to delete line and output result

I have a text file and manage to find the line through output comparison. With the book title and author name. awk -F ':' '/KungFu Feet|Chuck Norris/' test.txt Kungfu Feet:Chuck Norris:12:1:1 Is ...
3
votes
2answers
375 views

How to convert a .txt subtitle file to .srt format?

I have a subtitle file, it looks like this: 00:00:44:" Myślę, więc jestem".|Kartezjusz, 1596-1650 00:01:01:Trzynaste Pietro 00:01:06:Podobno niewiedza uszczęśliwia. 00:01:10:Po raz pierwszy w ...
3
votes
5answers
91 views

Join two files, matching on a column, with repetitions

How can I get two files A and B, and out put a result like this: File A: 001 Apple, CA 020 Banana, CN 023 Apple, LA 045 Orange, TT 101 Orange, OS 200 Kiwi, AA File B: 01-Dec-2013 01.664 001 ...
2
votes
3answers
100 views

Free BSD/ Mac OS X Sed: Print regex match and the line 5 lines after the match

I thought it would be sed -n /mymatch/,+5p but I get expected context address. At this point I am just guessing commands, nothing works! I want to grab every line that matches the regex and print ...
0
votes
1answer
113 views

How to print gmail inbox messages

I want to print inbox messages one by one (like cat someFile) so that I can parse the message contents. something like printGmail InboxMesg1 | grep some | sed some Using curl I can access the ...
3
votes
1answer
53 views

Expanding comma-separated list into separate lines

I have a file with some data in this form: Prefix text: First Name, Second Name, Third-- The prefix differs by line. The number of names varies from one to several. The suffix (-- in the example) ...
2
votes
4answers
67 views

Sorting records by field backwards (not top to bottom)

I have a little script which does a bunch of mounting in preparation for backing up, or which just shows you all currently mounted partitions. I've made it so that it will sort the output on several ...
1
vote
1answer
59 views

removing data from txt file

I have a comma-delimited giant file that was exported from a database. The file has this format Date,n1,n2,n3,n4,n5,n6,color,type,reference 10-Jan-2000,344,13,21,2,55,3,"rose garden" ,3444,32 ...
2
votes
2answers
38 views

How do I use cut to separate by multiple whitespace?

I would like to get the last column of this sample: [ 3] 1.0- 2.0 sec 1.00 MBytes 8.39 Mbits/sec [ 3] 2.0- 3.0 sec 768 KBytes 6.29 Mbits/sec [ 3] 3.0- 4.0 sec 512 KBytes 4.19 Mbits/sec ...
4
votes
2answers
107 views

tail -f, but with line numbers

I'm trying to see how many times foo bar appears in /var/log/foo.log within an arbitrary amount of time on a remote server, but nothing that I've tried so far has worked. I've already got a timer ...
2
votes
3answers
42 views

To assign variable from a file separated by delimiter

I have a text file with datas as below separated by delimiter: 1000:Tom:user1 1002:Jerry:user2 1003:Victor:user3 1004:Jason:user4 I need to run a while loop (any loop) to fetch details from first ...
2
votes
2answers
53 views

Process each line from text file and delete related files from disk

I used the find command to find several files and folders and output them to a text file like so: nohup find /oradba -name '*soapr*' 2>/dev/null >find_soapr_db40.txt & So instead of ...
8
votes
1answer
79 views

Filter the clipboard content without using an intermediate file

I'm trying to modify the text that is in my clipboard, remove certain lines of that contains some strings and then get the output for further manipulation (I prefer coping to my clipboard only if I ...
5
votes
1answer
99 views

How to append multiple lines to a file without last newline?

I have a very long string that is split in chunks. I want to append them to a file without putting the newline character using bash. Example: First append cat >> abc.sh << EOL echo "bla ...
-1
votes
1answer
87 views

extracting ip address from a file [closed]

I'm trying to extract all the IP address from a file. FILE *>i85.95.2.160/29 172.16.24.4 0 200 0 64515 4445 15412 i *>i85.112.80.0/28 172.16.24.4 0 200 ...
1
vote
3answers
58 views

Adding and removing text from the same place on every line of an emacs buffer

This is in a similar vein to my earlier question: Removing the last n characters from every line in an emacs buffer. The comments made there all apply here. Suppose I want to either insert the same ...
0
votes
2answers
71 views

Print 2nd and 7th lines to a file using `sed`

I have a directory of .txt files. For each, i'd like to grab the 2nd and 7th line, and add those lines to another file, which I've arbitrarily named list.index. I'm not sure how to tell sed about the ...
2
votes
3answers
51 views

Extracting part from a file column

I want to extract part from a log file column which is like this: xx.xxx.xx.xx#59796: Edit: This is the actual log line: Jan 10 17:38:11 server named[747]: client 21x.x0.x8x.xxx#40649: view ...
2
votes
2answers
80 views

Show a sliding window of output from a program

I want to run a command (rsync -av) and only display 10 lines of output at any time. I have found a similar post, but I don't want to redirect it to a file. I want the output to be displayed ...