Manipulation of text by programs, scripts, etc.

learn more… | top users | synonyms

1
vote
1answer
43 views

Replace case-insensitive string in all folder files

I'm trying to replace a string with another in all files in dir. Trying to use this command sed 's/foo/bar/' * But it replaces foo and not fOo or Foo and others. how to find "foo" ...
1
vote
1answer
57 views

Single record of a file getting splitted over multiple lines

We are using flat text files to store in teradata table after transformations from Informatica. The file contains 14 fields (~ separated). But as the records are getting scattered over multiple lines ...
2
votes
2answers
74 views

How to extract the positive numbers in the first column from an output as in the question?

I am running Ubuntu 14.04.1 LTS 64-bit with Bash 4.3.11(1)-release I have a program called harminv producing output as follows: $ h5totxt hsli0.126.h5 | harminv -vt 0.1 -w 2-3 -a 0.9 -f 200 # ...
2
votes
2answers
43 views

How do I extract the content of quoted strings from a file

have a file which looks like this: anm="12S123S" family="" type="Std" product="" name="nand" allow="yes" anm="12321 S=" family="" type="Std" product="" name="dan" need="No" anm="SWED" family="" ...
1
vote
1answer
54 views

print second column in transposed row

I have a space separated table (as in this example): A 101 B 92 102 C 83 93 103 D 74 84 94 104 ︙ I would like to obtain this: sample A with the associated value and all the second column ...
2
votes
5answers
74 views

Add every two consecutive lines in a column and divide it with the former and Print

I have a input file of almost 2000 lines. I have to do a calculation and print it in the third column. Example of what I have to do with the Input File: n ID1_1 n/(n+k) k ID1_2 Input File: 10 ...
0
votes
1answer
43 views

Loop for's result in a variable for sql query

I have this part of my script for i in `cat ${RELLIST}` do for mr in `cat ${i}/DBA/install.list` do grep -i "'dba_" ${i}/DBA/sql/${mr}/*.tab 2> /dev/null | sed 's/^.*://' #grep -i ...
1
vote
1answer
34 views

What does >… mean in vim and how to substitute a single whitespace for it?

I know that when :set list is used whitespace is shown as .. But what does it mean when a whitespace symbol is preceded by >, e.g. >., >.., or >...? And how do I remove it and leave only ...
1
vote
2answers
32 views

How do I transform array output sorted in lexicographic order to numeric order

I have a text file that consists of data in this form: int array[0] int array[1001] int array[1002] int array[1003] int array[2] int array[2001] int array[2002] ... In other words, the numbers ...
0
votes
3answers
45 views

Split the file comparing two fields in the files

Have two files which need to compare the first two columns. Sample input file1: |CID|SID|order|orderdate|confirmdate |151244127820|7177638911583| 2|2015-02-04 07:14:44|2015-02-04 07:15:32 ...
4
votes
3answers
93 views

How can I find and replace only if a match forms a whole word?

My script is: n="y" while [ "{n}" = "y" ] if [ $n == "n" ]; then break; fi echo "n is $n" do read -p "Enter the word to find = " word read -p "Enter word to replace = " replace ...
1
vote
2answers
163 views

extract blocks of text from xml file

I have a file with these contents: <People> <Person> <Name>Joe Blogs</Name> <Address>55 Oxford St</Address> ...lots of other properties ...
2
votes
2answers
116 views

Delete rows from a simplified CSV where some columns matching specific pattern

I have the following simplified CSV file (no separators or newlines embedded in fields): ...
1
vote
3answers
161 views

Compare two .csv files

I have two files: 'file1' has employee ID numbers. 'file2' has the complete database of the employees. Here is what they look like: file1 123123 222333 file2 111222 Jones Sally 111333 Johnson ...
0
votes
3answers
62 views

How can I output sample rows from a .csv?

I'm doing a lot of data merge printing and would like to be able to output, say, 10 samples from a large database. My goal would be to extract the 10 rows spread evenly from the supplied .csv ...
2
votes
6answers
115 views

Scripting to split a single CSV row into multiple

I need a script that can take a CSV file with that has a column of semicolon-delimited attributes, and break it into multiple rows to normalize that multi-valued attribute? Here is the scenario: ...
1
vote
3answers
71 views

Filter csv file based on extended column values

I have the following csv file: ID,PDBID,FirstResidue,SecondResidue,ThirdResidue,FourthResidue,Pattern RZ_AUTO_1,4tov,1404,1405,1518,1519,CG/AA Canonical ribose-zipper ...
2
votes
2answers
38 views

how to print the closest column when looking for a specific strings

I have a file with around 12 000 lines. ...
0
votes
2answers
47 views

How to add a column if line contains string?

If I have a tab-delimited file containing the following lines: Sample1-Baboon 1 2 3 4 Sample2-Human 1 2 3 4 Sample3-Human 1 2 3 4 Sample4-Baboon 1 2 3 4 How can I add a new column ...
0
votes
1answer
29 views

Filter by column value and write to file

I have a table like this: BB1520_v1.0_0103 exon 3329 3601 BB1520_v1.0_0103 exon 3650 4040 BB1520_v1.0_0089 exon 1 862 BB1520_v1.0_0089 exon 1312 ...
1
vote
1answer
54 views

sed regex not working for hidden characters

I have a list that has for some reason got some non alphanumeric characters appended to the end. I would have thought it would be a simple case of running : sed 's/[^A-Za-z0-9]$//' To replace ...
0
votes
5answers
62 views

How to Read string from file and compare to each line of second file

I'm making the post OS installation script that will include the Hardening section . And in this hardening section it will read the Kernel parameters from file A and compare to file /etc/sysctl.conf ...
-1
votes
1answer
39 views

How to add a line number follow by a tab to a text file

I tried using nl -ba -s '\t' full_media > full_media2 But it really did not add the tab, but it added actually the "\t" text after the line number. The file im editing is a CSV file and here by ...
1
vote
1answer
131 views

grep (/sed/awk) month range

I've looked around for solutions to this, but it's sort of difficult to search for the solution to such a specific problem (lot of words to put in Google). So, I have a file with data that looks like ...
-1
votes
1answer
53 views

Script to `grep` a file with variable [duplicate]

while read wholeline do echo ${wholeline} --outputs John Jones #grab all the lines that begin with these values #grep '^John Jones' workfile2 --works, prints contents of workfile2 where the ...
0
votes
2answers
53 views

How to run a script to get certain contents of a file

So say a file has the login data for a bunch of users: bobby activityname 2014-05-12(bobbymiller@gmail.com) bill activityname 2014-06-12(bobbymiller@gmail.com) bobby activityname ...
0
votes
0answers
48 views

Bash - Convert escape character colored text to BBCode

I want to convert a terminal escape-character colored text file to BBCode colored text. For this i've created an Android logfile with logcat -Cd > /sdcard/logcat.txt. The -C switch adds the color ...
2
votes
4answers
72 views

Substitute all occurrences of pattern with first matched group

I have to go from this: text match$something_here and match$xxx blablabla text match$something_else_here and match$xxx blablabla ... to this: text match$something_here and match$something_here ...
2
votes
4answers
77 views

Tool for escaping all quotes in a string/file

I have a weird problem that I don't have a good way around. My company, because reasons, stores scripting code in a database table. Don't ask. Whenever I have to insert a new version of the code, I ...
5
votes
3answers
111 views

Split an input for different command and combine the result

I know how to combine the result of different command paste -t',' <(commanda) <(commandb) I know pipe same input to different command cat myfile | tee >(commanda) >(commandb) Now ...
1
vote
2answers
58 views

Multiple grep commands with sort

I have a text file containing a list of student grades: Andrew cooking A Andrew cooking B Andrew computing F Steven java C Steven python E Ben C++ D Ben python C Jamie java E Jamie C++ E Jamie C++ B ...
1
vote
2answers
42 views

Generate Sequence number for each line at the end .. modified t he post can someone help&suggest on this

I am new to linux I have got this query sed '/./=' abc.txt| sed '/./N; s/\n/, /' >> as.dat source file has 3 rows like below a b c When use the following command it gives me result like ...
7
votes
8answers
621 views

Using 'grep' to find lines that contain all of three specified characters in any order [duplicate]

We have a document which contains lines and we have to find if [a|b|c] appears at least once in every line no matter which order. For example: Input: abc bca cab hhfdhdhfabjfdjdjff acjfdjdfjdf ...
3
votes
6answers
144 views

redirect output to specific line number

I want to grep certain lines by keyword and redirect the output to specific line number of an existing file. Command grep "key" temp_file >> desired.txt What I need is that I can append the ...
0
votes
1answer
78 views

Sum two columns from different files

I have two files like this: File1: -54.005 -67.405 0.1 -54.05 -68.7833 0.2 -54.05 -68.7333 0.3 File2: -54.005 -67.4050 70 12.7 -54.05 -68.7833 167 12.5 -54.05 -68.7333 205 12.6 ...
1
vote
1answer
33 views

Merge columns with the same value with awk

It is possible merge columns with the same value in awk? Something like if awk '$1 = $2' then merge I got this AZ14962 AZ14962 ctg_ 104 AZ14962 AZ14962 ctg_ 4 AZ14962 AZ14962 ctg_ 61 ...
0
votes
2answers
69 views

Using AWK to Print Output of Combination of Unspecified Number of Files

My objective is to move into a subdirectory, call it SUB, which has many text files with 5 columns each and always the same number of rows. I want one large 'sum' file that adds all columns from 2-5, ...
20
votes
2answers
1k views

Pipes, how do data flow in a pipeline?

I don't understand how the data flows in the pipeline and hope someone could clarify what is going on there. I thought a pipeline of commands processes files (text, arrays of strings) in line by line ...
1
vote
2answers
54 views

add a new line to a delimited file

So I have a file name test.txt inside that file I have about 20 lines of text that are delimited by pipe | Example: John|freshman|seatle|math|4|fulltime Bob|senior|Tacoma|biology|4|part-time I ...
3
votes
2answers
179 views

Find the line number which contains the pattern with custom delimiter

As in example I'm trying to get line numbers which contains the pattern. My pattern contains slashes so I wanted to add custom delimiter. This simple one works: sed -n '/file/=' temp.txt Using ...
0
votes
1answer
35 views

Compare two files [closed]

I want to compare two files in which the data is delimited by a pipe |. Both the files have the same results but only few fields might vary. File 1: A|B|C|D File 2: A|B|F|D I want to compare ...
2
votes
3answers
134 views

Writing a program for editing .txt data - Python or Unix?

I only have little experience in programming, and I'm currently working on improving my skills. Basically, I need to write a program, that can do some specific processes to some data in a .txt file. ...
1
vote
2answers
68 views

Copy the first 2 lines from a file and then remove the lines

I need to process a file using a shell script. The file arrives with the data as well as some embedded information releavant to how it is processed. The file is in the following format; ...
3
votes
5answers
208 views

How to replace only the third occurrence of the file in unix using only sed command?

How to replace the third occurrence of the string in the file using sed command. Example: Change only the third occurrence of is to us in the file. My input file contains: hai this is linux. hai ...
1
vote
3answers
54 views

Display distinct values of a list and number of occurences

I have a program that writes on the standard output a list of string values, one per line, and I would like to display in real time the list of distinct values along with the number of occurences for ...
3
votes
3answers
60 views

Unix: Add unique names after signs in text

I have a file looking like the following: abcdefghijklmnopqrst // abcdefghijklmnopqrst // abcdefghijklmnopqrst I use the following code to insert a line with // to the beginning of the document (as ...
3
votes
7answers
435 views

can we print the last word of each line in linux using sed command?

suppose , if there is a file consisting of following lines , if they are 12345 567 7878 66 er3 t45t y6y46y 4y6 y656y y5y 46y6 65y7 y66uyuy yy46y6y The output has to look like: 66 ...
1
vote
3answers
82 views

Replace date with current date

I have a file abc.fil whose content is in the format yyyymmdd.xyz.doc. I want to cat that file abc.fil such that I get the output as 20150127.xyz.doc. Can any one please help me to resolve this ...
2
votes
3answers
122 views

Replace multiline string in files

I have a number of files I want to update by replacing one multi-line string with another multi-line string. Something along the lines of: * Some text, * something else * another thing And I want ...
2
votes
4answers
61 views

Removing an empty line and the line above it

I have a large .fna file containing millions of DNA sequences. Due to some write errors after truncating my primers, I'm left with some sequence headers with no sequence following it. An example is ...