Manipulation of text by programs, scripts, etc.

learn more… | top users | synonyms

5
votes
2answers
91 views

Concatenating fields from lines with different numbers of fields

Given input like this: x y a b c t p q w w t a b c d p q r I'd like to concatenate fields from field 3 up to but not including the last one. If field 3 is the last field, I'd like to insert a ...
1
vote
2answers
542 views

using awk to create a LaTex table from my data in a text file

So I have used awk a few times to add or remove text from each line of a text file. What I would like to learn how to do now is to add some characters to each line in multiple places. What this is ...
5
votes
3answers
504 views

removing redundancy from output columns

What is a *NIX way of removing redundancy in a case where I have pairwise comparisons like these in two columns A B B A A C A D C A D A B C C B A B and B A ...
4
votes
5answers
682 views

How to print all the lines not containing a string along with adjacent lines?

I'm trying to wrap my mind around this task. I want from the following output, to exclude all the lines that contain Auto-installed: 1 and the 2 lines before. I was naively thinking that grep -v -B2 ...
8
votes
6answers
12k views

How to remove blank lines from a file in shell?

I want to remove all empty lines from a file. Even if the line contains spaces or tabs it should also be removed.
1
vote
1answer
60 views

List the lines with the highest scores

I have the following problem concerning the sorting of a content of a file. I have a file called points.txt and it contains the following table of data: Jim:4 Tim:8 Lin:6 Tom:1 Esko:2 Jaska:5 Erkki:7 ...
3
votes
2answers
2k views

Print some fields of /etc/passwd

I have a question concerning the usage of wild cards in Linux. Here I have some lines from a long list of lines in the /etc/passwd -file. root:x:0:0:root:/root:/bin/bash ...
1
vote
3answers
113 views

a string representation using regex (modified)

I would like to use regular expression to represent a pattern with the following type of characteristics 1) It includes a continuous sequence of digit values. The length of this sequence it at least ...
1
vote
3answers
283 views

Extracting mail addresses

I have a file that contains lines like this: user=<xxxxxxx@asd.com.tr>, user=<yyyyyyy@asd.com.tr>, user=<zzzzzzz@asd.com.tr>, user=<ccccccc@asd.com.tr>, ...
2
votes
4answers
4k views

Count tabs per line in text file with utils

I have a tab-tabulated file. I would like to check if every line has the same number of tabs. For first step, I'd like to print the number of tabs for each individual line. I've tried grep -o '\t' ...
1
vote
1answer
100 views

How to add parens around text

How can I use sed to replace text and also append to end of line for the match in question? I have a sed file which is doing many substitutions. One of them changes ~= to match( using /=\~/ ...
4
votes
3answers
353 views

how to emulate “replace a b b a” with a sed?

I am trying to emulate a replace command which swaps two characters, with a sed script, but the obvious choice: sed -e 's/a/b/g' -e 's/b/a/g' does no work. Can you point me how to do it correctly? ...
6
votes
5answers
959 views

Sorting blocks of lines

I have a file that contains 4n lines. Here is an excerpt from it containing 8 lines 6115 8.88443 6116 6.61875 6118 16.5949 6117 19.4129 6116 6.619 6117 16.5979 6118 19.4111 6115 8.88433 What I ...
2
votes
5answers
309 views

Read contents in a line in bash

I have a file with contents in this order which is separated by space 4096 1230 4560 1024.... n How can i read each number and pass it on to another command, eg divide each number by 1024.
3
votes
1answer
631 views

Strip most frequent words from text

I have a simple problem, but unfortunately I don't know where to even start (I'm just starting out). So, what I want to do is ultimately increase my vocabulary. I got the idea to strip the most ...
6
votes
1answer
3k views

How to grep all lines where starting line ends with a pattern and ending line ends with another?

I want to start grepping from the lines that has CK at the end of line and stop grepping when the line has D at the end. I tried grep "$CK" "$D" file..txt, but it didn't work. Input: kkkkkkkkkkk ...
8
votes
2answers
607 views

Merge three file in to a single master file by excluding the header in 2nd and 3rd files

I have three file with n number of lines as given below sample1.txt ------------------------------ Date Time Name ------------------------------ 2013/10/12 12:09:09 Krish ...
0
votes
2answers
86 views

Extracting information between tags [duplicate]

I have a text file, the following shows a sample of file contents: 1234 A novel homeodomain-encoding gene is associated with a large CpG island interrupted by the ...
4
votes
6answers
7k views

Remove lines that do not start with a pattern from a given set of patterns

I have a file that contains data like this: report aaaaaaaa - .. -th bbbbbbbbb -to ccccccccc .. --. Question: I want to remove any line that does not start with following strings : report ...
3
votes
2answers
142 views

Extract information from a text file

I have a text file, i want to search for tags such as the following: <category="SpecificDisease">Type II human complement C2 deficiency</category> <category="Modifier">Huntington ...
4
votes
3answers
97 views

Fast access to often used document

For several applications I maintain my personal cheatsheets. I use them mostly for vim, latex, vim latex-suite and bash. For example, when I forget a key combination in vim, I browse to the ...
7
votes
1answer
5k views

How to merge two files in corresponding row?

Now,I have two files: aaaa.txt: a=0; b=1; c=2; bbbb.txt: d=3 e=4 f=5 I want to merge aaaa.txt and bbbb.txt to cccc.txt. cccc.txt as follow: a=0;d=3 b=1;e=4 c=2;f=5 So, what can I do ...
4
votes
2answers
3k views

grep one line before the match plus the match

zzzzzzzzz aaaaaaaaa bbbbbbbbb & ccccccccc & ddddddddd hhhhhhhhh eeeeeeeee fffffffff & ggggggggg & in the above line, what i want is to grep/sed/awk (any method is fine) line that ...
7
votes
5answers
3k views

A better paste command

I have the following two files ( I padded the lines with dots so every line in a file is the same width and made file1 all caps to make it more clear). contents of file1: ETIAM...... SED........ ...
1
vote
1answer
533 views

Insert a space before the first comma in a csv file

I have a huge CSV file which contains 100000 rows. However, I need to do some processing and so for that I need to have a single space before the first comma. My current data is something like below. ...
0
votes
1answer
2k views

Awk with multiple delimiters

I am trying to execute awk on something like this : <tag x="whatever" y="http://mysite.com/somewords/word1/word2/word3/word4"> I would like to extract word2 which comes forever after word1, ...
2
votes
3answers
2k views

compare files line by line and create new one bash programming

I have two text files. File 2 has logs over 1,000,000. File 1 has IP addresses line by line. I want to read file 2 lines and search these lines in file 1, I mean: file 1: 34.123.21.32 45.231.43.21 ...
3
votes
3answers
14k views

Insert text in specific lines of a file?

How can insert some text in specific lines of a file? What is most simplest method that I can use? (bash, sed, awk?) What I want to do, might be simple, but I don't know where I should start. It's ...
11
votes
4answers
946 views

Shuffle file randomly with some additional constraints

I have a huge music playlist and, while some artists have many albums, others have just one song. I wanted to sort the playlist so the same artist won't play twice in a row, or his songs won't end up ...
2
votes
1answer
30 views

Replace output occurrences

Let's say I have this output: 0001 0002 0003 0004 0005 and I want to replace the first two zeros (00) with (01). What is the simplest way to achieve this?
3
votes
2answers
3k views

How to find frequency of occurrences of strings contained in a file?

I have a file that contains a list of URLs of the form EDIT http://www.google.com/absd/siidfs/kfd837382$%^$&, www.google.com, google.com yahoo.com/list/page/jhfjkshdjf... I ...
10
votes
1answer
997 views

generate a hyperlinked table of contents and insert into existing PDF

I have an existing PDF (without any corresponding source file), and a text file which is a list of items referencing sections within the PDF file, along with a page number for each item. I would like ...
4
votes
3answers
387 views

OS X/Linux one-liner/script to find the largest recurring group of lines in a text file?

I have a log containing an execution trace where there is infinite recursion eventually terminating when the stack is too deep. There are enough lines and valid included recursion within the larger ...
-1
votes
1answer
110 views

remove apostrophes and other marks files from apostrophes

How to remove apostrophes and other marks such as (Parkinson’s and Parkinson's ) from all text files in a directory and save the changes in each one of the files as in the directory
7
votes
4answers
29k views

How to find and replace string without use command Sed?

As we all know, sed is greatly efficient to find and replace string, for example find 'a' and replace it to 'b': sed 's/a/b/g'. Is it possible to do this with other command or shell script instead ...
4
votes
3answers
2k views

using sed to replace pattern with hash values

I want to search the file and replace specific pattern with its hash (SHA1) values. For example, let file.txt has the following content: one S56G one two three four five V67X six and I want to ...
12
votes
4answers
2k views

How to report “sed” in-place changes

When using sed to replace strings in-place, is there a way to make it report the changes it does (without relying on a diff of old and new files)? For instance, how can I change the command line ...
1
vote
1answer
9k views

C shell script : how do I compare two strings?

I'm a C shell user. I've two files. file 1: A B C D E file 2: A C B D E I want to compare only A-A, B-C, C-B, D-D, E-E using a for loop. If the strings match, then print "match". The output ...
3
votes
3answers
647 views

awk read error bad address

I have a file that is roughly 30GB. I am using, awk ' $1 <= 2000 ' myfile.txt >> myfile_new.txt But, I get this error after a while - awk: read error (Bad address) I guess this is ...
0
votes
2answers
514 views

Formatting quotes from the fortune to use it with conky

I'm using fortune cookie outputs in my conky script. My fortune output is in the following format: Text1 : Some message including( line break tags, e.g. and comma(,),=,semicolon(;) like ...
4
votes
2answers
479 views

Bash-script. Shift seconds

Maybe somebody can help me. In bash I don't know how to do that. I need to do a bash-script. At stdin I have .srt file of subtitles in this format: num HH:MM:SS,SSS --> HH:MM:SS,SSS text line 1 ...
4
votes
1answer
73 views

Extract data from two files

I have two files as given below Output1.csv 201319107648361,12:27:04,12:27:14,0:0:10 secs 201319109089878,12:27:04,12:27:15,0:0:11 secs 201319108968961,12:27:15,12:27:54,0:0:9 secs ...
1
vote
1answer
495 views

Binary search for a failing line

git has bisect run to figure out which revision an error was introduced. I have a big file (100GB) and at least one of the lines is bad, but the program I have to check it with will not tell me which ...
5
votes
2answers
2k views

Lowercase all but the first (Uppercase) letter from UPPERCASE in cyrillic

To make all letters a lower case except the first letter. The first letter would look like "Uppercase" after I changed (from UPPERCASE in cyrillic). The rest (not UPPERCASED) leave unchanged. I'm ...
3
votes
2answers
4k views

Show only part of the line found by grep (but not the part that matches the pattern)

In ifcfg-eth0, I managed to grep "DEVICE=eth0" by using grep "DEVICE=" ifcfg-eth0 But how can I grep pattern to show only "eth0"?
4
votes
5answers
3k views

How to align fstab entries easily

Editing /etc/fstab with my text editor makes all the spacing go out of alignment. I could go through the file and insert/delete spaces to line everything up, but I'm looking for a more automated ...
2
votes
5answers
8k views

Sum the values in a column except the header

I have a file as given below -------------------------------------------------------------- Name_Customer Item_Purchased Item_Amount Credit ...
1
vote
3answers
717 views

Find matches for the 5th columns in a pipe delim files

File 1: Connect|20130320000023|UTC|PPP|test3@test3.co.uk|test1@test1.co.uk|0BCBE578| File 2: Connect|20130320000023|UTC|PPP|test1@test1.co.uk|test1@test1.co.uk|0BCBE578| ...
3
votes
4answers
3k views

Align text to center with padding on both sides

Is there an easy way to align text to the center with padding on both sides, with the column width being the longest line from the input? For example, this: aaa bbbb c ddddd ee will turn into this ...
4
votes
2answers
19k views

Removing text between two specific strings

I have a file as below: mime PI Name: ISHO SUCCESS RATE RT, Value: 95.663826 scr PI Name: RRC Access Failures due to UU, Value: 0.13394141 prog PI Name: RRC Access Failures due to UU, Value: ...