Manipulation of text by programs, scripts, etc.

learn more… | top users | synonyms

4
votes
3answers
150 views

Frequency of words in non-English language text: how can I merge singular and plural forms etc.?

I'm sorting French language words in some text files according to frequency with a focus on insight rather than statistical significance. The challenge is about preserving accented characters and ...
-4
votes
2answers
53 views

Querying names of a text document

Print the lines that contain male firstnames in a file named muser.txt. Print the firstnames of all males from Italy. Print the number of different firstnames of all males from Italy. ex of doc: ...
0
votes
3answers
76 views

Linux, printing specific parts of a text document

I want to print, for each line of a given document, the first names and the ages of people (which are entries of the document). What linux command would I use? Hopefully based around grep using a regex ...
2
votes
2answers
371 views

Remove all lines in file A which contain the strings in file B

I have a CSV file users.csv with a list of userNames, userIDs, and other data: username, userid, sidebar_side, sidebar_colour "John Lennon", 90123412, "left", "blue" "Paul McCartny", 30923833, ...
6
votes
2answers
450 views

Number of comma-separated fields in a text file

I'm trying to build an awk statement to read this file: A 1,2,3 * A 4,5,6 ** B 1 B 4,5 * and build a file like this: A 1,2,3 * 3 1 0.333 A 4,5,6 ** 3 2 0.666 B ...
1
vote
3answers
138 views

How do I Copy and Paste lines between a start and end keyword?

I have two text files, and I want to copy a bunch of lines from one to the other. File one has a list of packages, and I want to copy it to list two. This list of packages isn't at the start of file ...
1
vote
4answers
99 views

Bash one liner to change configuration parameters

I have a config file with following structure. ValueOne = 1 ValueTwo = 2 ValueThree = 3 I want a one liner bash script to find ValueTwo and change the value to 22222. Any idea? Not School Thing
0
votes
1answer
295 views

Parse lines of output from bash loop

I'm trying to parse the output of commands run in a bash loop. Here is an example: $ for i in `git log --format='%H'`; do echo $i ; git branch --contains $i; done | head -n 8 ...
1
vote
3answers
70 views

Explicit range beween values

Is there any clean way of calculating the range between elements of the second column of a text file using some bash magic? (I'm currently doing this using Python). Input: File 1 A 1-5 A ...
1
vote
2answers
46 views

Have grep output spaced for each file

When I do grep on a bunch of files, and several file have the word I am looking for, the output is hard to read as it's one after the other. Is there anyway to insert additional \n's after the ...
2
votes
4answers
134 views

Use `cut` to extract a list from /etc/passwd

How could I use the cut command to extract a list of usernames and login shells from the /etc/passwd file, where the resulting usernames and login shells are separated by a single space? Sort the ...
3
votes
1answer
90 views

Convert json numbers to strings in the shell

When parsing json, the command-line tool jshon converts numbers to scientific notation, and sometimes tries to round them. To avoid these problems, I want jshon to consider these numbers as strings. ...
5
votes
3answers
234 views

How to extract URIs from files

How can I extract from a certain number of files all the URIs like magnet: file:? I'd like to have one URI per line as the output. I'd like to use bash to achieve this, but any suggestion is welcome. ...
3
votes
1answer
31 views

Calling latex from vim stopped working

When I try to compile my latex document from within vim by issuing: :w|!lualatex % (leaving out the w command does not solve the problem) it compiles but it does not save any files. If I call lualatex ...
4
votes
3answers
133 views

Search and replace strings that are not substrings of other strings

I have a list of replacements like so: search_and -> replace big_boy -> bb little_boy -> lb good_dog -> gd ... I need to make replacements for the above, but at the same time avoid ...
2
votes
3answers
170 views

bash - extract filenames from html file containing multiple links

I have downloaded an html file autogenerated by a script on a webpage. The file contains multiple links, including links to images I am trying to extract the full names of the images, for example ...
2
votes
3answers
60 views

How to add a new line for a missing entry between two fields

I have this file. As you can see in column1, entries TEST01N03, TEST01N06, and TEST01N18 are missing (total of 18 entries). Is there a way to check the fields of this column and fill in the missing ...
3
votes
2answers
135 views

Can we find page count of a file in Unix/Linux?

I want to count number of pages in the file /var/log/messages, so that I can use pr command to start printing from the page I want to. For example, suppose that /var/log/messages file contains 300 ...
2
votes
2answers
51 views

Merging files by rows

Is it possible to copy the whole rows of File1 in a new File3 following the instruction given by File2 by using a simple bash script using sed or awk? File1: /*two or more columns*/ AC 456324 DC ...
3
votes
1answer
292 views

merge csv files by first column

I have 3 csv files like this. csv 1: 1,aaaa,bbb,2014-04-01 2,qwe,rty,2014-04-03 3,zxc,cvb,2014-04-05 csv 2: 2,j,k,2014-04-01 3,a,s,2014-04-04 5,g,h,2014-04-08 csv 3: 2,a,s,d,f,g,2014-04-01 ...
2
votes
1answer
121 views

Compare the values of a column with all the values in another column

I have 2 input files. Each line of File1 should be compared to each and every line of File2. Logic is: If Column1 of File1 does not match Column1 (all the values under it) of File2, then print the ...
0
votes
2answers
1k views

iconv illegal input sequence- why?

While trying to convert a text file into its ASCII equivalent, I get error message that iconv: illegal input sequence at position. Command I use is iconv -f UTF-8 -t ascii//TRANSLIT file The ...
0
votes
2answers
38 views

How to format the output of free/sar/such commands?

I need to get some specific columns from the output of free command. The way I'm following doesn't seem good, could anyone please suggest me a better way? bash-3.2$ free -gotsi total ...
3
votes
2answers
376 views

tr complains of “Illegal byte sequence”

I'm brand new to UNIX and I am using Kirk McElhearn's "The Mac OS X Command Line" to teach myself some commands. I am attempting to use tr and grep so that I can search for text strings in a regular ...
6
votes
4answers
177 views

How to insert lines with numbers from X to Y into another file after line Z?

I created two files: echo -e "1\n2\n3\n4\n5" > 123.txt echo -e "a\nb\nc\nd\ne" > abc.txt I want to get the file 123.txt with the following contents: 1 2 3 b c d 4 5 So in other words ...
0
votes
2answers
79 views

How can I extract a specific string from a file?

I am having difficulties parsing my data. In the file shown below, I want to get the characters after BIG**20021208*00001**. I tried sed '/BIG.20021208.00001\**/!d;s///;s/\*.*//' but it doesn't work. ...
6
votes
7answers
603 views

Removing a specific line from a file

Somewhere in the middle of my CSV file is this line: Products below this line are out of stockNumber, month, year, reference, store Note:Number, month, year, reference and store are the CSV fields. ...
2
votes
2answers
84 views

Removing lines with a single common field

I have a tab separated file that looks like this: 123 some text 123 some different text 334 some other text 341 more text and I want to do two things. One is to order everything numerically ...
3
votes
2answers
81 views

The reverse of `tree` - reconstruct file and directory structure from text file contents?

We all know we can use tree to get a nicely formatted text "visualization" of the structure of a directory; say: $ tree -spugD /usr/include/boost/accumulators/numeric/ ...
4
votes
3answers
252 views

How can I delete a trailing newline in bash?

I'm looking for something that behaves like Perl's chomp. I'm looking for a command that simply prints its input, minus the last character if it's a newline: $ printf "one\ntwo\n" | ...
2
votes
2answers
70 views

How can I compare two files based on the value of the first column?

I want to compare two files based on the value of their first column: file1 aaa 23 3 bbb 56 6 ccc 45 2 file2 bbb m a aaa a m ccc m m The output file should be: aaa a m bbb m a ccc m m ...
3
votes
2answers
185 views

How to interleave the lines of two or more files?

I have tried to get the 2 files output in a single file. I have tried below command: cat file1 file2 but here file2 data is appending to file1. What I need is file1 1st line followed by file2 ...
2
votes
2answers
131 views

Printing lines with similar text together

I am currently working on analyzing the performance of an application. I've starting by looking at the logs generated by our application and identifying the business logic tasks that are taking more ...
2
votes
1answer
49 views

Extracting lines around keys from a file

I have a large file with around 15 million lines. The odd lines contains the keys (but not only the keys) and the even lines are the data. That is, the file is something like: WRTZ Tyrosine ...
3
votes
3answers
351 views

Sort comma-separated fields on each line by numeric value

I tried this: $ echo "2,3435,1" | sort -n 2,3435,1 $ sort -t',' -n test_sort.txt kill,gill,burger 110,20,30,13 $ cat test_sort.txt 110,20,30,13 kill,gill,burger Why doesn't my sort command work? ...
2
votes
1answer
475 views

Convert xml subtitle file to srt format

I have a xml subtitle that looks like this: <?xml version="1.0" encoding="utf-8" ?><transcript> <Item from="1.16" duration="4.68"><![CDATA[(Dong-hyuk is coming to see you now.) ...
1
vote
1answer
52 views

Retrieving fasta sequences using bed file information from locally installed file

I have a .bed file containing around 30000 rows for which I have the sequences retrieved using fetch-sequences module of the rsat tool (http://rsat.ulb.ac.be/rsat/help.fetch-sequences.html#usage). ...
3
votes
3answers
233 views

String replacement in file

I have the following file: <?xml version="1.0" encoding="utf-8"?> <!--Generated by crowdin.net--> <string name="test" >- test</string> <string name="test" ...
0
votes
2answers
52 views

Insert single successive numbers at the beginning of each line in a text file with Vim

Assume I have a text file containing some strings: string_g string_r string_qqwer string_p string_f string_3qw string_e string_qw And I need to insert increasing or decreasing numbers before the ...
2
votes
1answer
72 views

Change a specific line using sed

I need to change a line in a Java file called DbConnection.java using the sed command. The line is: connection = DriverManager.getConnection("jdbc:mysql://**localhost**/**bajaj**","**root**", ...
4
votes
2answers
254 views

Shell script (bash) to add or replace the license text contained within .c , .h and makefiles?

I have a set of *.c , *.h and Makefiles in a folder where some files contain the license text and some files don't have any license text. So I need a shell script where I can add license text if the ...
2
votes
1answer
61 views

set flag to remove duplicate records from a file based on some values of the record

I have a file having the following records (pipe delimited): SK200-10|ALBUMIN-SK|INS|SEKURE ALBUMIN 2 X ...
5
votes
2answers
101 views

Grepping for a block of text with parts that can be optional

I have multiple entries that describes an event in a very large log file, say A.log. I would like to do two things with the event entries in the log file: Count the number of occurrences of each ...
4
votes
4answers
228 views

Removing lines that are common to the outputs produced by two commands

Say, I have two commands that return some text. For example: $ ./c1 /usr/bin/foo /usr/bin/bar /usr/bin/baz $ ./c2 /usr/bin/foo /usr/bin/qux /usr/bin/buzz /usr/bin/bar I want to remove the duplicate ...
1
vote
1answer
97 views

Fastest way of find and replace in large file with replacing filed present in another file

I have two files. File1 is a csv with 60 fields: 111,Check1|^/h1/h2/h3,22062014184500,20,0..... 111,Check2|^/h43/h40/h9,22062014184500,4,..... 111,Check3|^/h1/h3/h4,22062014184500,0,0,..... File2 ...
1
vote
7answers
152 views

Get text between two special characters in a line using shell

I have a line which comes from a file via: cat aaa.txt | grep "HIJK" Output gives: HIJK="My name is HIJK" I need to get the text My name is HIJK into a variable in a shell script. I have tried ...
3
votes
6answers
80 views

Collapse some repeated parts of successive lines

I have data in notepad like this: 4480-1 4480-2 4480-3 4480-15 4581-1 4581-2 4581-3 4581-4 Can we do it using for loop sort of things? With sed? My required output is ...
0
votes
2answers
35 views

names that occur the largest number of times in a data set

users directory ---------------------- hjsyeh hellodir ; awgshd newdir; hjsyeh hidir; jkdhjk welcomedir the above given are set of users and the directories they created .How can I ...
2
votes
2answers
298 views

How to parse multiple string patterns in linux shell script

I am new to linux shell and I had a hard time thinking of the solution to this. Can somebody help how to parse multiple strings (data) in a file? For below file, I want to parse all the values next to ...
6
votes
3answers
90 views

Print a multiline variable in a column

I have a script that collects values from a cluster. The values, in some cases, have multiple lines. I have a printf format that specifies how the data should be positioned however, it doesn't take ...