Manipulation of text by programs, scripts, etc.

learn more… | top users | synonyms

3
votes
3answers
148 views

How can I delete text before a delimiter in Linux

I need to delete text from a line before delimiter. For example: (123434): hello::{apple,orange,mango}. I need to delete text before first :. i.e. (123434). Is there any command in linux to ...
1
vote
4answers
50 views

Increment a date in a file

I have a file with single line of text inside.How can I modify its content without using the vi editor.Below is the example: PROCESS DATE =2014-08-01 I want to increment the "2014-08-01" by the ...
2
votes
3answers
140 views

Text File Renaming Using Script

I have thousands of text files inside a folder and they are named in a certain way. For example: Image_234_Data_7778_n0.txt Image_234_Data_7778_n1.txt Image_234_Data_7778_n2.txt ...
2
votes
6answers
547 views

Handling comma in string values in a CSV file [duplicate]

I have a comma-separated file which has numeric and string columns. String columns are quoted and can have comma in between the quotes. How do I identify the columns with FS =","? Sample records ...
1
vote
3answers
105 views

merging all files into one file?

I was trying to merge all the text files in to one file. All the text files has one column and equal number of rows. My data file names are data_1.txt, data_2.txt, data_3.txt, . . . data_n.txt. ...
5
votes
2answers
115 views

Extract tomcat parameters from ps output

I'm trying to write a little diagnostic tool and one of the functions I want from it is that I want to run is: ps axu | grep tomcat | grep -v grep | awk '{ print $<list of desired fields> }' ...
0
votes
2answers
100 views

Delete every number that comes before “=”

i have a very, very, very long list. Like over 830,000 characters. I will of course perform said command on a copy. There are multiple lines, like this hey1=lol hey2=lol hey3=lol I want that to ...
2
votes
2answers
142 views

Collate tab-delimited files

I want to compare the information from mapping files to retrieve crosslinked information in a tab-delimited file format. For example, one file contains COG0001 882.DVU3168 COG0002 883.DvMF_2502 ...
4
votes
5answers
147 views

Insert Lines after Multi-Line Regex Match

I've got an XML file I need to append to after a specific set of tags. Basically the file structure looks like this: <key>servers</key> <dict> ... Server Details ... </dict> ...
0
votes
3answers
46 views

Delete everything not matching this pattern 'something.swf' in txt file

Can sed or awk do this, what I want do to is delete everything not matching this pattern 'something.swf' It doesn't need to be these commands. And im kind off new the text processing via the terminal ...
2
votes
3answers
81 views

How to convert html entities to readable text?

I want html number entities like &#x119; and want to convert it to real character. I have emails mostly from linkedin that look like this: chcia&#x142;abym zapyta&#x107;, czy ...
4
votes
2answers
173 views

Convert file with integers written in ascii to binary file of integers

I have a file that contains a long list of integers written in ascii separated with newlines, like such: -175 2 -19345 345592 -45 2355 etc... I want to convert this file into a "binary" file ...
1
vote
4answers
133 views

Bash: Convert Multi-line output to Single line

I am getting a multi-line output as follows: Actual Output: GenuineIntel,Intel(R) Xeon(R) CPU E5-2687W 0 @ 3.10GHz,CPU0,64,EM64T Family 6 Model 45 Stepping 7,(null),3093,0 GenuineIntel,Intel(R) ...
-3
votes
2answers
126 views

Extract certain column from CSV files [closed]

I have a directory with a group of CSV files. All files have the same column headers. I need to extract values in a certain column from all files. The common part in files names is ...
0
votes
1answer
36 views

Delete all lines containing “..” and “@”

I have a long text file from a tcpdump. I need some of the information but most lines are just like this: .....@isa..g..3..2..2.@...3.3.2.1.12.. I want to use sed to search and replace all lines ...
2
votes
3answers
117 views

Replace a word after a significant line and white spaces (inline) using sed?

I have the following text: A Hello world B Hello world C Hello world I know that I can replace Hello by Hi using sed: sed 's/Hello/Hi/g' -i test but this replace each Hello ...
2
votes
3answers
34 views

Delete until given character starting from given string

root@server:~/foo$ cat INPUT.txt limit. Current u'^'ZO limit. Current usage'^'FR limit. Current us'^'AS limit. Current usage 95 %.'^'FR limit. Cur'^'AEW root@server:~/foo$ sed ...
4
votes
1answer
157 views

Using sed command convert only first character in specific line with uppercase

How can I convert the first character of each word in the line after gr-description: to uppercase? For example : $ cat /tmp/test1 groMail: test@xyz.com gr-description: abc-location-999 ...
1
vote
2answers
57 views

Delete all words surrounded by parentheses with sed

How can I do this in sed, I have a long long list of things and some are surrounded by a parentheses. lol xml lol lol (347) idiot (67) How can delete (347) and (67) but for a huge list with numbers ...
1
vote
1answer
73 views

Replace a string in a file and overwrite the file

I wrote my script using the the directory ~/deleted (it's a long script). after using ~/deleted for the whole script. I want to let the user choose the directory by putting the path in ~/.rm.cfg. My ...
0
votes
1answer
65 views

Print lines if pattern is not matched

Print lines of File 1 only if they don't contain patterns specified in File 2 Exemple: $ cat File_1 L1 pattern1 L2 pattern pattern2 L3 pattern3 pattern L4 pattern4 $ cat File_2 ...
1
vote
3answers
47 views

Print all lines between a searched pattern of a text file to different file

I have a file called test.txt that contains: ....Request.... asaksa sda dsad dsad ....Request... 21mklk nnm212 mkmr543 849238 ....Request... 4392840 kndska 94i0-jkfjdk smdla .....Request.. 839281 ...
2
votes
2answers
90 views

Remove rows from tab delimited files based on a common column with another file

I am having trouble processing a tab delimited file based on the common column Column_4 with another file. One file is likely to be very small (less than 100 rows), the second one, however, will be ...
2
votes
5answers
226 views

extracting/copy ranges of columns from large data file

I was trying to prepare data files from one large file by extracting columns and pasting into a new file. The number of columns of the given data is very large. I want to prepare a number of data ...
4
votes
2answers
186 views

Joining two files

I have a 2 files with the following: File1.txt A 1 B 2 C 5 Z 3 File2.txt A 4 B 7 C 10 D 11 What I would like to do is create something like A 1 4 B 2 7 C 5 10 D - 11 Z 3 - Is there a ...
5
votes
4answers
220 views

How to replace m part of Lines with n part of Lines?

I have a file "test" like this: RewiteEngine On RewriteCond %{REMOTE_ADDR} !^192\.168\.1\.7$ [NC] RewriteCond %{REMOTE_ADDR} !^192\.168\.1\.8$ [NC] RewriteCond %{REMOTE_ADDR} ...
9
votes
4answers
255 views

Store output of a command into a ring-buffer

I've got a long-running command that generates a lot of output on stdout. I'd like to be able to retain, for instance, only the last three days or the last gibibyte (avoiding cutting lines in the ...
1
vote
2answers
188 views

Extracting text string from file, assigning them to variables then writing them to another file?

Basically, its a game server (minecraft) and I have a situation where I am attempting to automate new world generation. Basically, I have a file that gets generated happily which has this snippet in ...
1
vote
2answers
82 views

How to add words to an existing column

I have a tab delimited file with 10 columns and in one of the columns (with about 40 million rows), I would like to add a word before the existing entry in each row (same word in each row!) and a ; ...
2
votes
3answers
64 views

Shifting command output to the right

Say I'm writing a .bashrc file to give me some useful information in my login terminals and I'm telling it to run the cal command (a nice one). How would I go about shifting the calendar produced to ...
0
votes
3answers
54 views

String index processing

My input file looks like: #key string pos(string) key1 AA000AA000000AAA0A 2, 3, 18, 12 key2 00A00AAA000AAAA00A 3, 18 And I'd like to add a new column at ...
4
votes
4answers
279 views

Convert key=value blocks to CSV

I am trying to transpose a content of a file into another. I am new to shell scripting any help would be really appreciated Input file Test.txt: HLRSN = 3 IMSI = 404212109727229 KIVALUE = ...
0
votes
3answers
69 views

Remove all capital letters from a particular column in a CSV file

I am having a CSV file containing 3000+ lines in the following format: R1C1, ABC123abc, R1C3 R2C1, DEFG456defg, R2C3 R3C1, HIJKL789hijkl, R3C3 I need to remove all the capital letters from the ...
5
votes
3answers
61 views

How to put the output of the trailing lines of grep in on-line?

Assuming that I have this file: Thu May 8 15:32:07 2014 User-Name = "Mark" Framed-IP-Address = 0.0.0.0 Acct-Status-Type = Interim-Update Acct-Input-Octets = 95684 ...
1
vote
2answers
50 views

Calculate relative positions in strings

Sample input file: #name complete(cs) len(cs) simple(ss) len(ss) position(ss) NAME1 A0AAA000AAA00A 14 AAAAAAAA 8 4,6 NAME2 AAAA0AA00000A 13 AAAAAAA 7 7 ...
0
votes
3answers
53 views

replace comma in specific column value

I have a file which contains 6 columns, each field separated by "I" the second last column contains amount field. 140121059|01/01/201400:00:45|[1390]|[387]|17.64|10 ...
1
vote
0answers
164 views

How to create CSV file without unnecessary spaces

I use the xls2csv binary in order to translate XLS doc to CSV in my linux red-hat machine, example: ( from man page ) xls2csv -x "1252spreadsheet.xls" -b WINDOWS-1252 -c "ut8csvfile.csv" -a UTF-8 ...
2
votes
2answers
78 views

Replace column entry with sum of other column entries

Let's say I have a file that contains these lines: 02.03.14 14.50 14.50 0.00 Desc 02.03.14 17.00 0.00 17.00 Desc 01.03.14 2.82 1.68 calc Desc 02.03.14 1.04 0.00 ...
6
votes
5answers
189 views

Remove backslash + newline sequences

We are having a file in Linux which contains one line per record, but problem comes when the line contains some new line characters. In this case, the backslash is appended at the end of line and the ...
5
votes
4answers
86 views

Search a pattern and print preceding lines starting with another pattern

I need to grep for a pattern and once I find it, I need to look for another pattern preceding that and print all those lines in between those. Better example: 1 a 2 a 3 a 4 a 5 a 6 b 7 c 8 d 9 xyz ...
1
vote
1answer
47 views

How to align columns of multiple lines at a fixed distance with vim

Here is another one of my vim question (They seem to become my trademark here.). I have a really long file basically consisting of lines of the following form: ;07 :ANZ textv S9_1 9_1_K3_2 ...
4
votes
2answers
39 views

How to sort a collection of lines from different files?

I want to sort second line in each file by second column, and also print name of the corresponding file. I am doing it like this - rm tmp; for filename in file*; do num=`head -2 $filename | tail ...
6
votes
7answers
312 views

Count total number of lines before/after a pattern match

I am having a long list of IP addresses, which are not in sequence. I need to find how many IP addresses are there before/after a particular IP address. How can I achieve this?
2
votes
0answers
55 views

Shell scripts for the database logs

The actual log file will be like this $ cat file1.log time=2014-07-23 23:56:28 GMT, user=[unknown], db=[unknown], host= pid=28254 LOG: time=2014-07-23 23:56:28 GMT, user=portalman, db=ss, ...
4
votes
6answers
218 views

reading a list of lines from a file writing output to a utility

I have a file with a list of lines I want to read from another file. I want to output those lines to a utility (grep) that lets me read the entire line and pull information from it. The file with the ...
2
votes
2answers
59 views

Format output to a specific line length

So I have some output that has very long lines, and I want to have it formatted to lines no more than 80 columns wide, but I don't want to split words because they are in the column limit. I tried ...
0
votes
3answers
49 views

Reading through a csv file

I have a file Books.txt which contains the names of books and the people who have borrowed it . BOOKXXXX is the name of the book and XXXX are digits. Here is an example. BOOK2592,1,2,3,4,5,22 ...
5
votes
2answers
190 views

Comment all lines from last commented line to line with 'foo'

Consider a text file users.txt: #alice #bob charlie dotan eric I need to comment everything from (exclusive) the last commented line until (inclusive) dotan. This is the result: #alice #bob ...
2
votes
2answers
115 views

Merging columns from two separate files

How to create a new file merging selective columns from two separate files using awk? Without messing up the elements orders of BOTH files. Exemple: File 3 may contain column 1,2,3 from File 1 and ...
2
votes
2answers
138 views

Format the content of the file using Perl

I am new to Perl programming and I am stuck with this problem for some time. Can you please help me in solving it? I have a file with the following format: >Country1 Aus trali a >Country5 ...