Manipulation of text by programs, scripts, etc.

learn more… | top users | synonyms

1
vote
2answers
25 views

Calculate relative positions in strings

Sample input file: #name complete(cs) len(cs) simple(ss) len(ss) position(ss) NAME1 A0AAA000AAA00A 14 AAAAAAAA 8 4,6 NAME2 AAAA0AA00000A 13 AAAAAAA 7 7 ...
0
votes
3answers
18 views

replace comma in specific column value

I have a file which contains 6 columns, each field separated by "I" the second last column contains amount field. 140121059|01/01/201400:00:45|[1390]|[387]|17.64|10 ...
1
vote
0answers
47 views

How to create CSV file without unnecessary spaces

I use the xls2csv binary in order to translate XLS doc to CSV in my linux red-hat machine, example: ( from man page ) xls2csv -x "1252spreadsheet.xls" -b WINDOWS-1252 -c "ut8csvfile.csv" -a UTF-8 ...
2
votes
2answers
64 views

Replace column entry with sum of other column entries

Let's say I have a file that contains these lines: 02.03.14 14.50 14.50 0.00 Desc 02.03.14 17.00 0.00 17.00 Desc 01.03.14 2.82 1.68 calc Desc 02.03.14 1.04 0.00 ...
6
votes
5answers
151 views

Remove backslash + newline sequences

We are having a file in Linux which contains one line per record, but problem comes when the line contains some new line characters. In this case, the backslash is appended at the end of line and the ...
4
votes
3answers
57 views

Search a pattern and print preceding lines starting with another pattern

I need to grep for a pattern and once I find it, I need to look for another pattern preceding that and print all those lines in between those. Better example: 1 a 2 a 3 a 4 a 5 a 6 b 7 c 8 d 9 xyz ...
1
vote
1answer
23 views

How to align columns of multiple lines at a fixed distance with vim

Here is another one of my vim question (They seem to become my trademark here.). I have a really long file basically consisting of lines of the following form: ;07 :ANZ textv S9_1 9_1_K3_2 ...
4
votes
2answers
35 views

How to sort a collection of lines from different files?

I want to sort second line in each file by second column, and also print name of the corresponding file. I am doing it like this - rm tmp; for filename in file*; do num=`head -2 $filename | tail ...
6
votes
7answers
201 views

Count total number of lines before/after a pattern match

I am having a long list of IP addresses, which are not in sequence. I need to find how many IP addresses are there before/after a particular IP address. How can I achieve this?
2
votes
0answers
29 views

Shell scripts for the database logs

The actual log file will be like this $ cat file1.log time=2014-07-23 23:56:28 GMT, user=[unknown], db=[unknown], host= pid=28254 LOG: time=2014-07-23 23:56:28 GMT, user=portalman, db=ss, ...
4
votes
6answers
55 views

reading a list of lines from a file writing output to a utility

I have a file with a list of lines I want to read from another file. I want to output those lines to a utility (grep) that lets me read the entire line and pull information from it. The file with the ...
1
vote
2answers
33 views

Format output to a specific line length

So I have some output that has very long lines, and I want to have it formatted to lines no more than 80 columns wide, but I don't want to split words because they are in the column limit. I tried ...
0
votes
3answers
42 views

Reading through a csv file

I have a file Books.txt which contains the names of books and the people who have borrowed it . BOOKXXXX is the name of the book and XXXX are digits. Here is an example. BOOK2592,1,2,3,4,5,22 ...
5
votes
2answers
175 views

Comment all lines from last commented line to line with 'foo'

Consider a text file users.txt: #alice #bob charlie dotan eric I need to comment everything from (exclusive) the last commented line until (inclusive) dotan. This is the result: #alice #bob ...
2
votes
2answers
33 views

Merging columns from two separate files

How to create a new file merging selective columns from two separate files using awk? Without messing up the elements orders of BOTH files. Exemple: File 3 may contain column 1,2,3 from File 1 and ...
2
votes
2answers
104 views

Format the content of the file using Perl

I am new to Perl programming and I am stuck with this problem for some time. Can you please help me in solving it? I have a file with the following format: >Country1 Aus trali a >Country5 ...
4
votes
3answers
106 views

Frequency of words in non-English language text: how can I merge singular and plural forms etc.?

I'm sorting French language words in some text files according to frequency with a focus on insight rather than statistical significance. The challenge is about preserving accented characters and ...
-4
votes
2answers
49 views

Querying names of a text document

Print the lines that contain male firstnames in a file named muser.txt. Print the firstnames of all males from Italy. Print the number of different firstnames of all males from Italy. ex of doc: ...
0
votes
4answers
71 views

Linux, printing specific parts of a text document

I want to print, for each line of a given document, the first names and the ages of people (which are entries of the document). What linux command would I use? Hopefully based around grep using a regex ...
2
votes
2answers
125 views

Remove all lines in file A which contain the strings in file B

I have a CSV file users.csv with a list of userNames, userIDs, and other data: username, userid, sidebar_side, sidebar_colour "John Lennon", 90123412, "left", "blue" "Paul McCartny", 30923833, ...
6
votes
2answers
186 views

Number of comma-separated fields in a text file

I'm trying to build an awk statement to read this file: A 1,2,3 * A 4,5,6 ** B 1 B 4,5 * and build a file like this: A 1,2,3 * 3 1 0.333 A 4,5,6 ** 3 2 0.666 B ...
1
vote
3answers
79 views

How do I Copy and Paste lines between a start and end keyword?

I have two text files, and I want to copy a bunch of lines from one to the other. File one has a list of packages, and I want to copy it to list two. This list of packages isn't at the start of file ...
0
votes
4answers
31 views

Bash one liner to change configuration parameters

I have a config file with following structure. ValueOne = 1 ValueTwo = 2 ValueThree = 3 I want a one liner bash script to find ValueTwo and change the value to 22222. Any idea? Not School Thing
0
votes
1answer
43 views

Parse lines of output from bash loop

I'm trying to parse the output of commands run in a bash loop. Here is an example: $ for i in `git log --format='%H'`; do echo $i ; git branch --contains $i; done | head -n 8 ...
1
vote
3answers
60 views

Explicit range beween values

Is there any clean way of calculating the range between elements of the second column of a text file using some bash magic? (I'm currently doing this using Python). Input: File 1 A 1-5 A ...
1
vote
2answers
29 views

Have grep output spaced for each file

When I do grep on a bunch of files, and several file have the word I am looking for, the output is hard to read as it's one after the other. Is there anyway to insert additional \n's after the ...
1
vote
4answers
77 views

Use `cut` to extract a list from /etc/passwd

How could I use the cut command to extract a list of usernames and login shells from the /etc/passwd file, where the resulting usernames and login shells are separated by a single space? Sort the ...
3
votes
1answer
63 views

Convert json numbers to strings in the shell

When parsing json, the command-line tool jshon converts numbers to scientific notation, and sometimes tries to round them. To avoid these problems, I want jshon to consider these numbers as strings. ...
5
votes
3answers
209 views

How to extract URIs from files

How can I extract from a certain number of files all the URIs like magnet: file:? I'd like to have one URI per line as the output. I'd like to use bash to achieve this, but any suggestion is welcome. ...
3
votes
1answer
22 views

Calling latex from vim stopped working

When I try to compile my latex document from within vim by issuing: :w|!lualatex % (leaving out the w command does not solve the problem) it compiles but it does not save any files. If I call lualatex ...
4
votes
3answers
98 views

Search and replace strings that are not substrings of other strings

I have a list of replacements like so: search_and -> replace big_boy -> bb little_boy -> lb good_dog -> gd ... I need to make replacements for the above, but at the same time avoid ...
2
votes
3answers
45 views

bash - extract filenames from html file containing multiple links

I have downloaded an html file autogenerated by a script on a webpage. The file contains multiple links, including links to images I am trying to extract the full names of the images, for example ...
2
votes
3answers
56 views

How to add a new line for a missing entry between two fields

I have this file. As you can see in column1, entries TEST01N03, TEST01N06, and TEST01N18 are missing (total of 18 entries). Is there a way to check the fields of this column and fill in the missing ...
3
votes
2answers
105 views

Can we find page count of a file in Unix/Linux?

I want to count number of pages in the file /var/log/messages, so that I can use pr command to start printing from the page I want to. For example, suppose that /var/log/messages file contains 300 ...
2
votes
2answers
33 views

Merging files by rows

Is it possible to copy the whole rows of File1 in a new File3 following the instruction given by File2 by using a simple bash script using sed or awk? File1: /*two or more columns*/ AC 456324 DC ...
2
votes
1answer
64 views

merge csv files by first column

I have 3 csv files like this. csv 1: 1,aaaa,bbb,2014-04-01 2,qwe,rty,2014-04-03 3,zxc,cvb,2014-04-05 csv 2: 2,j,k,2014-04-01 3,a,s,2014-04-04 5,g,h,2014-04-08 csv 3: 2,a,s,d,f,g,2014-04-01 ...
2
votes
1answer
70 views

Compare the values of a column with all the values in another column

I have 2 input files. Each line of File1 should be compared to each and every line of File2. Logic is: If Column1 of File1 does not match Column1 (all the values under it) of File2, then print the ...
0
votes
2answers
49 views

iconv illegal input sequence- why?

While trying to convert a text file into its ASCII equivalent, I get error message that iconv: illegal input sequence at position. Command I use is iconv -f UTF-8 -t ascii//TRANSLIT file The ...
0
votes
2answers
27 views

How to format the output of free/sar/such commands?

I need to get some specific columns from the output of free command. The way I'm following doesn't seem good, could anyone please suggest me a better way? bash-3.2$ free -gotsi total ...
2
votes
2answers
60 views

tr complains of “Illegal byte sequence”

I'm brand new to UNIX and I am using Kirk McElhearn's "The Mac OS X Command Line" to teach myself some commands. I am attempting to use tr and grep so that I can search for text strings in a regular ...
6
votes
4answers
142 views

How to insert lines with numbers from X to Y into another file after line Z?

I created two files: echo -e "1\n2\n3\n4\n5" > 123.txt echo -e "a\nb\nc\nd\ne" > abc.txt I want to get the file 123.txt with the following contents: 1 2 3 b c d 4 5 So in other words ...
0
votes
2answers
73 views

How can I extract a specific string from a file?

I am having difficulties parsing my data. In the file shown below, I want to get the characters after BIG**20021208*00001**. I tried sed '/BIG.20021208.00001\**/!d;s///;s/\*.*//' but it doesn't work. ...
6
votes
7answers
555 views

Removing a specific line from a file

Somewhere in the middle of my CSV file is this line: Products below this line are out of stockNumber, month, year, reference, store Note:Number, month, year, reference and store are the CSV fields. ...
2
votes
2answers
57 views

Removing lines with a single common field

I have a tab separated file that looks like this: 123 some text 123 some different text 334 some other text 341 more text and I want to do two things. One is to order everything numerically ...
3
votes
2answers
45 views

The reverse of `tree` - reconstruct file and directory structure from text file contents?

We all know we can use tree to get a nicely formatted text "visualization" of the structure of a directory; say: $ tree -spugD /usr/include/boost/accumulators/numeric/ ...
4
votes
3answers
185 views

How can I delete a trailing newline in bash?

I'm looking for something that behaves like Perl's chomp. I'm looking for a command that simply prints its input, minus the last character if it's a newline: $ printf "one\ntwo\n" | ...
2
votes
2answers
46 views

How can I compare two files based on the value of the first column?

I want to compare two files based on the value of their first column: file1 aaa 23 3 bbb 56 6 ccc 45 2 file2 bbb m a aaa a m ccc m m The output file should be: aaa a m bbb m a ccc m m ...
2
votes
1answer
102 views

How to interleave the lines of two or more files?

I have tried to get the 2 files output in a single file. I have tried below command: cat file1 file2 but here file2 data is appending to file1. What I need is file1 1st line followed by file2 ...
2
votes
2answers
126 views

Printing lines with similar text together

I am currently working on analyzing the performance of an application. I've starting by looking at the logs generated by our application and identifying the business logic tasks that are taking more ...
2
votes
1answer
37 views

Extracting lines around keys from a file

I have a large file with around 15 million lines. The odd lines contains the keys (but not only the keys) and the even lines are the data. That is, the file is something like: WRTZ Tyrosine ...