Manipulation of text by programs, scripts, etc.

learn more… | top users | synonyms

1
vote
1answer
117 views

Parse/Manipulate in awk

I am new to awk programming. I have a question on manipulating text file, which is required to draw certain network based images in a visualization software (Circos http://circos.ca) I have input ...
0
votes
0answers
74 views

Editing metadata with bash

I have a bunch of photos(jpeg, bmp, png) that I have arranged in such a manner that the metadata is in html form in a separate file. I would like create some script to read this html file and edit ...
2
votes
1answer
51 views

Text processing for words connected by hyphen

I have text files (*.txt) which I process, I want to find words connected by hyphen such as "drug-associated" or "drug-linked", "drug-related". I need your help to convert the first one to ...
0
votes
1answer
25 views

Python: Move lines backward in a textfile

Imagine a text file which contains random text and two unique markers 01 text text text 02 text text text 03 __DELETE_THIS_FIRST__ 04 text text text 05 text text text 06 text text text 07 text text ...
0
votes
4answers
80 views

How to extract the similarities between two strings

I have two strings in two different sentences: string 1: 30 mutation alanine for valine string 2: alanine at position 30 is there away to find the similarities between them as it is clear they ...
2
votes
4answers
125 views

Color matching but print other lines too [duplicate]

I have an alias: alias grep='grep --color=auto'. I wish to have some way to print not only matching row but all of them and just color matching patterns. I doubt is there an option to do that in the ...
1
vote
2answers
117 views

Issues Splitting CSV files

Im trying to split a csv file into multiple files using the below command..This command splits into 5 files but the data is truncated in one of the file? How do I fix this issue? >>split -n 5 ...
10
votes
9answers
1k views

How can I print the longest number in a string?

I am looking for a method to print the longest number in a string. E.g.: If I have the string 212334123434test233 how can I print 212334123434 ? Note: I am looking for the longest continuous ...
6
votes
4answers
633 views

We need a “C” list that contains all the elements of “A”, but doesn't have any of “B”

We have two lists. A bigger "A": A=`echo -e '1\n2\n3\n4\n5'` echo "$A" 1 2 3 4 5 and a smaller "B": B=`echo -e '1\n2\n3'` echo "$B" 1 2 3 Q: But we need a third list that contains all the ...
2
votes
3answers
45 views

way to select lines that show variations

I have a huge file that contains 80 genomes and five chromosomes. I was wondering is there is an easy way to select only the lines from ONE chromosome (for example 1) but only at the position showing ...
3
votes
3answers
134 views

Subsetting a file's contents based on value in a column

I have a text file as (for example): apple V$NFY_Q6_rc V=0.741 apple V$HOXA7_01_rc V=0.454 apple V$ALPHACP1_01_rc V=0.695 orange V$SP4_Q5 V=0.747 grapes V$SP1_Q2_01 V=0.677 grapes ...
1
vote
2answers
147 views

Remove just one word before pattern match in shell script

Input: int foo(int a, void * b, int c); Output: int foo(int, void *, int); Want to use "," as pattern to match. If "," pattern is matched then remove the previous word i.e. a, b & c.
1
vote
1answer
112 views

bash scripts - remove duplicate rows with smaller value

Here is an example file: 1 5 20 40 60 2 1 20 20 20 2 2 30 30 30 4 5 40 40 40 7 2 50 30 30 7 1 20 20 20 I would like to remove duplicate rows (with the same value in the first column) with a bigger ...
2
votes
2answers
46 views

How to format textutil -excludedelements

I use Geektool on the Mac to display various information on my desktop. One feature of Geektoool is the ability to use shell mode to launch custom scripts or commands. I am not versed in any type of ...
2
votes
3answers
829 views

Replacing the values in one file with the values in another file in bash

I am having a csv file naming List.csv in following format: Location,IP Address,Host Name,Domain,Domain Name, User Name,Manufacturer,Model,System Type, Serial Number, Operating System,RAM ...
1
vote
1answer
43 views

How to add a dot to particular lines in LaTeX-document?

I have a LaTeX-document where I have lines of the form \section*{3.1} \addcontentsline{toc}{section}{3.1} or in general \section*{x.y} \addcontentsline{toc}{section}{x.y} for x between 1 and 31 ...
3
votes
2answers
138 views

sed / awk / perl / etc: Move lines backward in file

Imagine a text file which contains random text and two unique markers 01 text text text 02 text text text 03 __DELETE_THIS_LINE_BEGIN__ 04 text text text 05 text text text 06 text text text 07 text ...
3
votes
3answers
425 views

How can I split a text file based on the content into multiple text files?

I have a text file called CAMS.txt that contains the following: 4153999999999991 4153999999999992 4153999999999993 4153999999999994 4801999999999991 4801999999999992 4801999999999993 I would like ...
0
votes
2answers
38 views

Sorting by Last Name in the Scond Column

I'm trying to sort a file that has entries that look like so: Essie | Vaill | 14225 Hancock Dr | Anchorage | AK | 99515 907-345-0962 Please note the "|" stand for tabs of ...
1
vote
5answers
258 views

Replacing backslashes with commas in Bash

I tried writing a script in which I am reading through a file line-by-line. I am supposed to replace a backslash (\) with a comma (,). The input file is as follows: 1,2,3,WORKSTATION1\A,5 ...
1
vote
2answers
123 views

How to multiply two columns in awk?

I want to multiply column 1 with column 2 (till end of file) in input file and should output 1 column and multiplied 3 column in separate file. input.txt 1 677679866 2 121867616 3 49413198 4 ...
1
vote
1answer
36 views

insert value at the end of every record based on unique IDs

I have 2 files , one with Delimited and other one is Positional file. These two files shares same unique ID's as shown below but both the files contains different information. del.txt ...
0
votes
2answers
136 views

Compare first column in different files

I want to compare one file with other files in the same directory. file1.txt contains: move 34#123# get 11#278 put 33#29#567#23 file1.txt should compare with files 1.txt and 2.txt. 1.txt ...
0
votes
1answer
44 views

How can I print a specific occurence of a range pattern?

I am trying to extract several data fields out of somewhat peculiarly formatted strings. Each string has a format similar to the following (spaces added for emphasis): 1 abcd 2 1 efg 2 hij 3 ...
5
votes
3answers
152 views

Make a directory for each row within the given text file?

I'm trying to write a bash shell script that will read the data from a given (as an argument) file, and for each row in the file, make a sub-directory under the given (as an argument) directory. The ...
7
votes
5answers
390 views

SED : Case Match Pattern Replacement

I have a source code spread across several files. It has a pattern abcdef which I need to replace with pqrstuvxyz. The pattern could be Abcdef (Sentence Case) then it needs to be replaced with ...
3
votes
2answers
84 views

Printing in awk up to specific line and 5 more results

I'm trying to print out all the lines up to the appearance of a particular song and 5 more songs with that. The problem i'm new to the shell what I have to fix in the bellow commands? RANK NAME ...
3
votes
5answers
154 views

How to prepend a particular text file contents to every text file in a directory and its subdirectories?

I hate to clutter my source code files with licensing/attribution comments but some times it is a requirement. So having dozens of source code files (organized in a tree of subdirectories) written I ...
3
votes
3answers
181 views

Rearrange text file by prepending data rows with header row

I have a file whose content looks like this: 2009 150 0 0 0 0000 75.316 0.0400390625 0.00007 0.00000 0.8980 76.216 0.0400390625 0.00007 1.00000 0.9046 77.217 0.0400390625 ...
3
votes
3answers
103 views

Append columns in a text file to after the final row

A smaller version of many data files I have looks like this: 0 0 0 0.05 9.6877884e-06 0.0024898597 0.1 4.2838688e-05 0.0049595502 0.15 0.00016929444 0.0074092494 0.2 0.00036426881 0.009839138 0.25 ...
1
vote
3answers
66 views

Print the number of values which have the special characters in them

I have a text file like the one below. I want to check the second column and print the count of values which have some special characters in them. 101,aaa,d01 102,*&%,d02 103,$%&,d03 ...
1
vote
3answers
70 views

Find records dated before today

how to find record form bellows data having "Last Date" before today's date. i.e : if today date is 09-04-2014 then finding record having "Last Date" before 09-04-2014 - Email ID : abc@domain.com ...
1
vote
2answers
830 views

Editing a Java properties file with Bash

I have the following Java properties file that is maintained by humans. # file.input = /very/old/name # file.input = /old/name file.input = /current/name file.input.default = /default/name ...
1
vote
5answers
125 views

How to remove lines shorter than XY?

I found a question about, how to remove lines longer then 2048 chars: How to delete line if longer than XY? Q: But how can I remove lines shorter then 4 chars? So remove lines that has 1 or 2 or 3 ...
1
vote
2answers
153 views

Bash: split list of files into different files?

I have several files in a directory like: data1.dat data2.dat .... data24678.dat To run another code, I need to have a text file with the names of the files but I need to split in 50 files per ...
0
votes
1answer
131 views

merge unsorted file to sorted file using shell command

I want to merge/join or produce a new merged file . Both files contain common IDs in the first column . File 1 looks like this ID Tb927.4.4670 Tb927.8.3630 Tb09.160.4310 Tb927.8.3650 File 2 ...
1
vote
1answer
33 views

Text pre-processing How to run many scripts at once

I have many scripts for text cleaning, is there a way to execute all of them at once instead of doing that one by one, for example I have : sed -i -r "s/[’'–]/ /g" *.txt sed -i 's/Aβ/ ABeta /g' ...
5
votes
4answers
487 views

How to grep blocks? Or I should use awk/ack?

Say I have a file with information as follows: ... Entry '234238': some text some text some text some text Entry '899823': some text some text some text Entry '234238': more text more ...
3
votes
2answers
63 views

find numbers in parentheses and round them

Say I have a file called tables.tex (lots of tables formatted for tex, including a preamble) with lines like this: some words (xyz, abc) & 0.00071 (0.07846) & 0.00411 (-0.13542) \\ some more ...
1
vote
1answer
107 views

Convert newlines to null-delimited when using tail

How can I change the output from tail to use null-terminated lines instead of newlines? My question is similar to this one: How to do `head` and `tail` on null-delimited input in bash?, but differs ...
8
votes
7answers
456 views

How can I identify lines in files over a certain length

I'd like to find lines in my code which exceed a certain length. My code is in multiple files. What's a good way to do this? I'd like to know the files and line numbers; content would be preferred, ...
3
votes
3answers
4k views

How to parse JSON with shell scripting in Linux?

I have a JSON output from which I need to extract a few parameters in Linux. This is the JSON output: { "OwnerId": "121456789127", "ReservationId": "r-48465168", "Groups": ...
1
vote
1answer
59 views

Comment (automatically) a LaTeX source file to make it more readable

I am looking for a simple way to do the following : Input : \begin{document} ... \section{} ... \subsection{} ... \subsubsection{} ... \section{} ... \end{document} Output : \begin{document} ... ...
3
votes
4answers
209 views

Split the input file with line breaks

I am trying to insert line breaks on a file based on the same names which happens to be the second field in my file. So, my input file is like below. 17412193|name1|organization ...
7
votes
1answer
51 views

How to display the non-sparse parts of a sparse file?

Imagine a file created with: truncate -s1T file echo test >> file truncate -s2T file I now have a 2 tebibyte file (that occupies 4kiB on disk), with "test\n" written in the middle. How would ...
3
votes
1answer
63 views

Find and Replace Recursive

I want to modify every file in a directory recursively. I want to replace: <?php with <?php /** * * Copyright (C) MyCompany, Ltd. - All Rights Reserved * Unauthorized copying of ...
2
votes
2answers
436 views

How to find and replace a field column value in UNIX

I have file with the value say which is delimited with | 123-|aaa|bbb|123|123.0|123-|123.01-|-123.02|123.03-|aaa|bbb|123-|aaa-|-bbb|cc-cc|123.04-|aa123-|123.05- I need to pick the column values ...
3
votes
4answers
144 views

Identifying genes from a list of genes

I have a gene list file. Some thing like this SWT21 SSA1 NRP1 EFB1 TFC3 MDM10 I have another file which also contains the names of these genes in my list along with other ...
2
votes
2answers
657 views

How to insert text after a certain string in a file?

Right now I'm using echo "Hello World" >> file.txt to append some text to a file but I also need to add text below a certain string let's say "[option]", is it possible with sed? EG: Input ...
2
votes
3answers
61 views

A shell tool to “tablify” input data containing ANSI escape codes

I have input containing ANSI color codes that I'd like to tablify. I want the output to remain colored, so the tablified output should maintain the ANSI color codes. Hence, naively stripping them away ...