Manipulation of text by programs, scripts, etc.

learn more… | top users | synonyms

0
votes
3answers
36 views

Delete everything not matching this pattern 'something.swf' in txt file

Can sed or awk do this, what I want do to is delete everything not matching this pattern 'something.swf' It doesn't need to be these commands. And im kind off new the text processing via the terminal ...
1
vote
3answers
39 views

How to convert ę to letter?

I want html number entities like ę and want to convert it to real character. I have emails mostly from linkedin that look like this: chciałabym zapytać, czy ...
4
votes
2answers
91 views

Convert file with integers written in ascii to binary file of integers

I have a file that contains a long list of integers written in ascii separated with newlines, like such: -175 2 -19345 345592 -45 2355 etc... I want to convert this file into a "binary" file ...
1
vote
4answers
91 views

Bash: Convert Multi-line output to Single line

I am getting a multi-line output as follows: Actual Output: GenuineIntel,Intel(R) Xeon(R) CPU E5-2687W 0 @ 3.10GHz,CPU0,64,EM64T Family 6 Model 45 Stepping 7,(null),3093,0 GenuineIntel,Intel(R) ...
-3
votes
2answers
56 views

Extract certain column from CSV files [closed]

I have a directory with a group of CSV files. All files have the same column headers. I need to extract values in a certain column from all files. The common part in files names is ...
0
votes
1answer
35 views

Delete all lines containing “..” and “@”

I have a long text file from a tcpdump. I need some of the information but most lines are just like this: .....@isa..g..3..2..2.@...3.3.2.1.12.. I want to use sed to search and replace all lines ...
2
votes
3answers
86 views

Replace a word after a significant line and white spaces (inline) using sed?

I have the following text: A Hello world B Hello world C Hello world I know that I can replace Hello by Hi using sed: sed 's/Hello/Hi/g' -i test but this replace each Hello ...
2
votes
3answers
29 views

Delete until given character starting from given string

root@server:~/foo$ cat INPUT.txt limit. Current u'^'ZO limit. Current usage'^'FR limit. Current us'^'AS limit. Current usage 95 %.'^'FR limit. Cur'^'AEW root@server:~/foo$ sed ...
4
votes
1answer
79 views

Using sed command convert only first character in specific line with uppercase

How can I convert the first character of each word in the line after gr-description: to uppercase? For example : $ cat /tmp/test1 groMail: test@xyz.com gr-description: abc-location-999 ...
1
vote
2answers
44 views

Delete all words surrounded by parentheses with sed

How can I do this in sed, I have a long long list of things and some are surrounded by a parentheses. lol xml lol lol (347) idiot (67) How can delete (347) and (67) but for a huge list with numbers ...
1
vote
1answer
61 views

Replace a string in a file and overwrite the file

I wrote my script using the the directory ~/deleted (it's a long script). after using ~/deleted for the whole script. I want to let the user choose the directory by putting the path in ~/.rm.cfg. My ...
0
votes
1answer
52 views

Print lines if pattern is not matched

Print lines of File 1 only if they don't contain patterns specified in File 2 Exemple: $ cat File_1 L1 pattern1 L2 pattern pattern2 L3 pattern3 pattern L4 pattern4 $ cat File_2 ...
1
vote
3answers
43 views

Print all lines between a searched pattern of a text file to different file

I have a file called test.txt that contains: ....Request.... asaksa sda dsad dsad ....Request... 21mklk nnm212 mkmr543 849238 ....Request... 4392840 kndska 94i0-jkfjdk smdla .....Request.. 839281 ...
2
votes
2answers
69 views

Remove rows from tab delimited files based on a common column with another file

I am having trouble processing a tab delimited file based on the common column Column_4 with another file. One file is likely to be very small (less than 100 rows), the second one, however, will be ...
2
votes
5answers
158 views

extracting/copy ranges of columns from large data file

I was trying to prepare data files from one large file by extracting columns and pasting into a new file. The number of columns of the given data is very large. I want to prepare a number of data ...
4
votes
2answers
174 views

Joining two files

I have a 2 files with the following: File1.txt A 1 B 2 C 5 Z 3 File2.txt A 4 B 7 C 10 D 11 What I would like to do is create something like A 1 4 B 2 7 C 5 10 D - 11 Z 3 - Is there a ...
5
votes
4answers
207 views

How to replace m part of Lines with n part of Lines?

I have a file "test" like this: RewiteEngine On RewriteCond %{REMOTE_ADDR} !^192\.168\.1\.7$ [NC] RewriteCond %{REMOTE_ADDR} !^192\.168\.1\.8$ [NC] RewriteCond %{REMOTE_ADDR} ...
9
votes
3answers
211 views

Store output of a command into a ring-buffer

I've got a long-running command that generates a lot of output on stdout. I'd like to be able to retain, for instance, only the last three days or the last gibibyte (avoiding cutting lines in the ...
1
vote
2answers
102 views

Extracting text string from file, assigning them to variables then writing them to another file?

Basically, its a game server (minecraft) and I have a situation where I am attempting to automate new world generation. Basically, I have a file that gets generated happily which has this snippet in ...
1
vote
2answers
66 views

How to add words to an existing column

I have a tab delimited file with 10 columns and in one of the columns (with about 40 million rows), I would like to add a word before the existing entry in each row (same word in each row!) and a ; ...
2
votes
3answers
51 views

Shifting command output to the right

Say I'm writing a .bashrc file to give me some useful information in my login terminals and I'm telling it to run the cal command (a nice one). How would I go about shifting the calendar produced to ...
0
votes
3answers
48 views

String index processing

My input file looks like: #key string pos(string) key1 AA000AA000000AAA0A 2, 3, 18, 12 key2 00A00AAA000AAAA00A 3, 18 And I'd like to add a new column at ...
4
votes
4answers
240 views

Convert key=value blocks to CSV

I am trying to transpose a content of a file into another. I am new to shell scripting any help would be really appreciated Input file Test.txt: HLRSN = 3 IMSI = 404212109727229 KIVALUE = ...
0
votes
3answers
64 views

Remove all capital letters from a particular column in a CSV file

I am having a CSV file containing 3000+ lines in the following format: R1C1, ABC123abc, R1C3 R2C1, DEFG456defg, R2C3 R3C1, HIJKL789hijkl, R3C3 I need to remove all the capital letters from the ...
5
votes
3answers
58 views

How to put the output of the trailing lines of grep in on-line?

Assuming that I have this file: Thu May 8 15:32:07 2014 User-Name = "Mark" Framed-IP-Address = 0.0.0.0 Acct-Status-Type = Interim-Update Acct-Input-Octets = 95684 ...
1
vote
2answers
46 views

Calculate relative positions in strings

Sample input file: #name complete(cs) len(cs) simple(ss) len(ss) position(ss) NAME1 A0AAA000AAA00A 14 AAAAAAAA 8 4,6 NAME2 AAAA0AA00000A 13 AAAAAAA 7 7 ...
0
votes
3answers
31 views

replace comma in specific column value

I have a file which contains 6 columns, each field separated by "I" the second last column contains amount field. 140121059|01/01/201400:00:45|[1390]|[387]|17.64|10 ...
1
vote
0answers
68 views

How to create CSV file without unnecessary spaces

I use the xls2csv binary in order to translate XLS doc to CSV in my linux red-hat machine, example: ( from man page ) xls2csv -x "1252spreadsheet.xls" -b WINDOWS-1252 -c "ut8csvfile.csv" -a UTF-8 ...
2
votes
2answers
70 views

Replace column entry with sum of other column entries

Let's say I have a file that contains these lines: 02.03.14 14.50 14.50 0.00 Desc 02.03.14 17.00 0.00 17.00 Desc 01.03.14 2.82 1.68 calc Desc 02.03.14 1.04 0.00 ...
6
votes
5answers
167 views

Remove backslash + newline sequences

We are having a file in Linux which contains one line per record, but problem comes when the line contains some new line characters. In this case, the backslash is appended at the end of line and the ...
5
votes
4answers
73 views

Search a pattern and print preceding lines starting with another pattern

I need to grep for a pattern and once I find it, I need to look for another pattern preceding that and print all those lines in between those. Better example: 1 a 2 a 3 a 4 a 5 a 6 b 7 c 8 d 9 xyz ...
1
vote
1answer
36 views

How to align columns of multiple lines at a fixed distance with vim

Here is another one of my vim question (They seem to become my trademark here.). I have a really long file basically consisting of lines of the following form: ;07 :ANZ textv S9_1 9_1_K3_2 ...
4
votes
2answers
38 views

How to sort a collection of lines from different files?

I want to sort second line in each file by second column, and also print name of the corresponding file. I am doing it like this - rm tmp; for filename in file*; do num=`head -2 $filename | tail ...
6
votes
7answers
242 views

Count total number of lines before/after a pattern match

I am having a long list of IP addresses, which are not in sequence. I need to find how many IP addresses are there before/after a particular IP address. How can I achieve this?
2
votes
0answers
38 views

Shell scripts for the database logs

The actual log file will be like this $ cat file1.log time=2014-07-23 23:56:28 GMT, user=[unknown], db=[unknown], host= pid=28254 LOG: time=2014-07-23 23:56:28 GMT, user=portalman, db=ss, ...
4
votes
6answers
79 views

reading a list of lines from a file writing output to a utility

I have a file with a list of lines I want to read from another file. I want to output those lines to a utility (grep) that lets me read the entire line and pull information from it. The file with the ...
1
vote
2answers
39 views

Format output to a specific line length

So I have some output that has very long lines, and I want to have it formatted to lines no more than 80 columns wide, but I don't want to split words because they are in the column limit. I tried ...
0
votes
3answers
46 views

Reading through a csv file

I have a file Books.txt which contains the names of books and the people who have borrowed it . BOOKXXXX is the name of the book and XXXX are digits. Here is an example. BOOK2592,1,2,3,4,5,22 ...
5
votes
2answers
183 views

Comment all lines from last commented line to line with 'foo'

Consider a text file users.txt: #alice #bob charlie dotan eric I need to comment everything from (exclusive) the last commented line until (inclusive) dotan. This is the result: #alice #bob ...
2
votes
2answers
53 views

Merging columns from two separate files

How to create a new file merging selective columns from two separate files using awk? Without messing up the elements orders of BOTH files. Exemple: File 3 may contain column 1,2,3 from File 1 and ...
2
votes
2answers
113 views

Format the content of the file using Perl

I am new to Perl programming and I am stuck with this problem for some time. Can you please help me in solving it? I have a file with the following format: >Country1 Aus trali a >Country5 ...
4
votes
3answers
119 views

Frequency of words in non-English language text: how can I merge singular and plural forms etc.?

I'm sorting French language words in some text files according to frequency with a focus on insight rather than statistical significance. The challenge is about preserving accented characters and ...
-4
votes
2answers
52 views

Querying names of a text document

Print the lines that contain male firstnames in a file named muser.txt. Print the firstnames of all males from Italy. Print the number of different firstnames of all males from Italy. ex of doc: ...
0
votes
3answers
72 views

Linux, printing specific parts of a text document

I want to print, for each line of a given document, the first names and the ages of people (which are entries of the document). What linux command would I use? Hopefully based around grep using a regex ...
2
votes
2answers
165 views

Remove all lines in file A which contain the strings in file B

I have a CSV file users.csv with a list of userNames, userIDs, and other data: username, userid, sidebar_side, sidebar_colour "John Lennon", 90123412, "left", "blue" "Paul McCartny", 30923833, ...
6
votes
2answers
222 views

Number of comma-separated fields in a text file

I'm trying to build an awk statement to read this file: A 1,2,3 * A 4,5,6 ** B 1 B 4,5 * and build a file like this: A 1,2,3 * 3 1 0.333 A 4,5,6 ** 3 2 0.666 B ...
1
vote
3answers
88 views

How do I Copy and Paste lines between a start and end keyword?

I have two text files, and I want to copy a bunch of lines from one to the other. File one has a list of packages, and I want to copy it to list two. This list of packages isn't at the start of file ...
0
votes
4answers
47 views

Bash one liner to change configuration parameters

I have a config file with following structure. ValueOne = 1 ValueTwo = 2 ValueThree = 3 I want a one liner bash script to find ValueTwo and change the value to 22222. Any idea? Not School Thing
0
votes
1answer
78 views

Parse lines of output from bash loop

I'm trying to parse the output of commands run in a bash loop. Here is an example: $ for i in `git log --format='%H'`; do echo $i ; git branch --contains $i; done | head -n 8 ...
1
vote
3answers
61 views

Explicit range beween values

Is there any clean way of calculating the range between elements of the second column of a text file using some bash magic? (I'm currently doing this using Python). Input: File 1 A 1-5 A ...