Manipulation of text by programs, scripts, etc.

learn more… | top users | synonyms

1
vote
0answers
37 views

How tcan I delete columns from a file without losing the format? [closed]

I have a file with this information: 1 2 3 4 5 O O O O O Eigenvalues -- -20.44628 -...
0
votes
1answer
26 views

How to move lines between patterns to the end of a file

I want to move several lines to the end of the file. Example: 1 2 3 4 5 6 blkn XYZ indf 7 8 9 10 11 lnkn ABCYDF knnfd 12 13 14 15 I want to move lines 6 to 11 to the end of the file. The patterns ...
3
votes
6answers
768 views

How do I remove a words ending in the letter S if duplicates without an S on the end are on the same list?

I have a large list of words. Many of the words are only different because they have the letter s on the end. If a word on the list is the exact same as another word on the list, except one of the ...
2
votes
1answer
73 views

How to sum numbers inside many 2D arrays

I need to sum numbers located in a file like this one: column1 column2 column3 row1 a(1,1) a(1,2) a(1,3) row2 a(2,1) a(2,2) a(2,3) row3 a(3,1) a(3,2) a(3,3) row4 a(4,1) a(4,2) ...
1
vote
3answers
38 views

How to delete whitespace surrounding equal signs in a string?

I'm trying to determine a way to delete leading and trailing whitespace surrounding the equal sign from a string. The whitespace may (or may not) be present. There may be 0, 1 or more equal signs. I'...
3
votes
4answers
95 views

All possible permutations of words in different files in pairs

I have multiple files, let's say file1, file2 etc. Each file has one word in each line, like: file1 file2 file3 one four six two five three What I want is to combine them in a new file4 in ...
-1
votes
1answer
23 views

Replace values with #<value># with date - '01-Jan-2015'

I want to replace values in file. Replace : values #startdate# with - '01-Jan-2015' values #code# with - 'SA','CA'
1
vote
1answer
45 views

Need to filter the data on the basis of two column having many to one relationship using awk

I have a big file having 50s columns and 100K of rows delimited by |. Now $2(col 2) has multiple type of $1(col 1) value which means col 2 will be repeated. So I have sorted the file. I need now to ...
-1
votes
2answers
56 views

how to change record or field separator of a file [closed]

How can I change the record separator of a file? For example, concatenating multiple lines into a single line, by changing record separator from newline to space. Another example is to split a line at ...
1
vote
3answers
58 views

Remove all the rows below a specific row number from one specific column

I've asked a question before on true row deletion, and got some great answers, but I'm now interested to know how you would remove all the rows below a specific row number from one specific column. ...
0
votes
1answer
24 views

extract two columns and add at a specific location to another file that is already full of columns

I have a file where I want to extract two columns and add at a specific location to another file that is already full of columns. Could I cat file 1 and make it a variable and then cat file 2 and ...
6
votes
4answers
71 views

True column deletion, not selecting all columns that you want

I know this is a simple question and in many ways has been answered before, however every use I've seen of awk, sed, grep etc have all been selecting and printing the columns you want. In a case where ...
2
votes
2answers
53 views

add a file name to lines in this file

I have fasta files, coming from different samples eg: 01.fasta, 02.fasta, etc. All of them contain OTU list with the following pattern: >OTU1 CCTACGGGCGGCTGCAGT >OTU2 CCTACGGGTGGCTGCAG I ...
2
votes
2answers
44 views

I would like to replace a pattern with another pattern found in the same line

I would like to replace the second tab-delimited field with the first underscore delimited field found in the same line Input Cho1_01:C4WMNACXX:1:250326554 1 2 4 2 -9 1 1 4 3 1 4 1 -9 4 1 4 1 3 ...
1
vote
0answers
21 views

Amend perl script so that words are matched on a word for word basis [migrated]

I have been using this perl script (thanks to Jeff Schaller) to match 3 or more words in the title fields of two separate csv files. Original question here: Matching 3 or more words from fields in ...
0
votes
1answer
28 views

How to insert fields from one file into fields of another file at particular location?

Is there any quick way to inserts fields from one file into another. For example, if the file contents of file1 and file 2 are like this : File1: ======= size1 size2 size3 File2: ======= char ...
1
vote
4answers
36 views

cut column 2 from text file

My text file has no delimiter to specify separator just spaces, how do I cut out column 2 to output file, 39 207 City and County of San Francisc REJECTED MAT = 0 78 412 Cases and ...
3
votes
1answer
56 views

Join arbitrary lines to output as one line

I want to join the 3rd and 5th line of a file to the output of a script without trailing new lines. Is it possible to implement this construction by using the code: { echo 'Some text' echo 'Some ...
1
vote
1answer
26 views

How to append the first line at the end of each line of a file with Sed?

I'm trying to append the first line of a file at the end of each line with a single sed command: Example: line 1 line 2 line 3 line 4 should become: line 1 line 1 line 2 line 1 line 3 line 1 line ...
4
votes
2answers
60 views

Find IP addresses visiting /page1 but not /page2 from nginx access logfile

I need help with selecting specific data's from log files. I need two scripts: I need to select all IP addresses that only visited /page1 I need to select all IP addresses that visited /page1 but ...
3
votes
2answers
45 views

Get a string and place it at the start of a line until the next string is found

I have a big non-standard xhtml file that I'm using sed (about 4 times) to iterate down to just the basics I need to throw the data into a MySQL database. My last bit I'm struggling with. The file is ...
3
votes
2answers
85 views

Organizing three dimensional data with awk/sed/perl

I have this file (a sparse matrix): PC.354 OTU1 6 PC.354 OTU2 1 PC.356 OTU0 4 PC.356 OTU2 7 PC.356 OTU3 3 I want an output like this (dense matrix -classic .biom table): OTU_ID PC.354 PC.355 PC....
1
vote
1answer
19 views

Library database

I have a database of readers and I would like to lend them a book and write it into the database. Example: Reader 753 Leonardo DaVinci davinci@gmail.com 321654987 Lended Books: grep "...
0
votes
2answers
29 views

Generating wordlist permutations from existing files

I have two wordlist files; the first contains: John Jerry Jim the second contains: one two three The files are several thousand lines each, not just the 3 words above. How can I generate ...
2
votes
2answers
42 views

Adding 'exception' words to the rule of perl matching titles script

I have been using this perl script (thanks to Jeff Schaller) to match 3 or more words in the title fields of two separate csv files as answered here: Matching 3 or more words from fields in separate ...
0
votes
3answers
61 views

How can I retrieve and compare two values from a file? [closed]

How can I retrieve and compare two values from the first two lines of a file? I have the following text: 05-24-2016, 2:59:32,0,0 05-24-2016, 2:59:37,0,0 05-24-2016, 2:59:42,0,0 05-24-2016, 2:59:47,0,...
0
votes
3answers
60 views

How to extract logs between the current time and the last 15 minutes

I want to extract the logs between the current time stamp and 15 minutes before and sent an email to the people configured. I developed the below script but it's not working properly; can someone help ...
2
votes
1answer
37 views

How to standardize the number of characters in a column?

I have a column (in my data-file database) where the number of characters is not standardized, and need to standardize. How can I do it? Example: Sample_ID 159 148 1110 1100 150 But my bank does ...
3
votes
4answers
57 views

Remove a certain word in a file

How do I remove just abcd11.gmail.com and *test1: from line 1 and abcd14.gmail.com and *fortest2: from line 2 and abcd19.gmail.com and *asatesteg: from line 2 from below leaving the remaining as is? I ...
0
votes
1answer
59 views

Using regexp_replace to replace single quotes around string but not within words e.g. can't

I'm using the below code from a bash script to replace single quotes from my title column in a postgres database with spaces. psql -U postgres -d my_database -c "UPDATE my_table SET title = ...
1
vote
1answer
41 views

Sort text by specific location on each line

What I want to achieve is sorting lines of text by a specific point in the text (the lines of text have no spaces or columns). I've worked out a solution to this, but I want to know if there is a more ...
3
votes
1answer
45 views

Substitute text fields with values from other file

Alright, I cannot figure out how to do this. I have a fileA which looks like this : ([7]RIMS_ID) : "CNR" (refGain_A[7]) ...
0
votes
0answers
24 views

copying every 2nd line starting from 1 [duplicate]

The below has actually been asked before, which I hadn’t realised. but out of curiosity, how would you go about it if you wanted not just odds or evens, for example every 5th line I'm trying to copy ...
1
vote
2answers
72 views

Removing spaces from the fields in a semicolon-delimited file using shell script

I have a csv file like this: 03139; 5; IT1234978208; 20150930 ; CTZ 13/31.12.15; 03137; 6; IT1234978206; 20151015 ; CTZ 13/31.11.18; .... .... 03134; 8; IT1234974406; ...
1
vote
2answers
60 views

find string from one file in another if not present then remove from original file

I'm trying to make a script that looks through each line of one file and if a line fails to match anywhere in any line of another text file then remove that line from the original file. so far I've ...
1
vote
1answer
53 views

How to find match with surrounding words?

I want to find match with surrounding words. I also want it to be fast. Here's what I've got: grep -o -P '(\w+ ){0,n}match( \w+){0,m}' n is number of words before the phrase, m is number of words ...
2
votes
4answers
56 views

Create text corresponding to multiple variable assignments from a single variable

I have a variable that has a value like this line=dog,/path1,/path2 What I want is to create a variable from the value; i.e dog should be the variable and want to get the below output: dog=/path1,/...
3
votes
1answer
45 views

Modify values in a column based on number of colons it contains

I would manipulate the contents of the second column based on the number of colons it contains. If the second field contains more than one colon then I would require the content before first colon ...
3
votes
5answers
315 views

Increment index in file

Alright, it seems I cannot find a way to do what I need. Let's say I have a text file A like this (freqBiasL2[27]) (SatBiasL1[27]) (defSatBiasL2_L1[27]) (defSatBiasSlope[27]) (...
3
votes
1answer
30 views

Use a list file to extract a second list from a master file (and comment out some lines)…HOW?

I have two files: master.tbl and sites.lst The file master.tbl is a list of sites, with their port configurations, port names, IP addresses, etc. There are lines containing site descriptions, ...
0
votes
1answer
26 views

Each line into a file (XML as part of file)

Here is a sample of the content in the file: <:abc>test file name <:/abc> XYZ 1122 XML content is pretty heavy and big. Under the directory I am working in I want to create a sub-...
0
votes
4answers
56 views

Duplicate the content of a file

I have a requirement where if a specific condition is met I need to duplicate the whole content of a file into the same file. Here is the file: $ cat in.txt ip 10.22.1.34 10.22.1.35 When a ...
0
votes
1answer
42 views

Remove lines from a file

In my output file I have the below lines included multiple times along with the required output. WARN: The method class org.apache.commons.logging.impl.SLF4JLogFactory#release() was invoked. WARN: ...
-1
votes
3answers
30 views

Count non-zero numbers for each row of a second column in a text file

I have a text file in that each line words are separated by commas, like this: 7022122465,0,\N,,0,2015-09-29 10:48:33 7022597642,0,\N,,0,2015-09-29 10:48:33 7022848906,0,\N,,0,2015-09-29 10:48:33 ...
0
votes
1answer
63 views

sed use from android shell

I am getting familiar with sed on the android terminal emulation: the version I am using the sed included in the toybox which is installed on my unrooted Sony device. sed --version (or toybox sed --...
1
vote
2answers
43 views

sed + add last word on each string only if word not appears

I have the following file: linuxA.mtt.corp linux3V linux4B linux2A linux5v.mtt.corp how to add the .mtt.corp on each string that not have this word?
0
votes
1answer
62 views

Matching 3 or more words from fields in separate csv files

I have two csv files: This is csv1: 11, The Sun Still Shines in Reading, 64312, 464566 13, You and Me Together Again London, 564564, 131355 12, What's the Story Now Your Gone, 4545646, 1124545 17, ...
4
votes
2answers
201 views

Display file text one page at a time, wait 20 seconds, then auto-advance

This question is for AIX 7 I have been working on a monitor which will display several pages of text. Initially the end user wanted a scrolling list for which I constructed this: IFS=''; cat plfeed ...
1
vote
1answer
36 views

find matching URLs in a line, then shorten them to the domain name

I am new to the Linux command line and need some help. I have a log file I am writing a script for so it only shows certain fields of output. The last bit I need is to shorten the URL's so that it ...
-3
votes
2answers
38 views

add a row from a file into another file as row

I have file1 whose content is like this : The 6 columns are obtained from 6 different hosts before presenting them using "-exec cp" command. This is just FYI. Now, I have a list of 6 hostnames (...