Manipulation of text by programs, scripts, etc.

learn more… | top users | synonyms

-1
votes
1answer
17 views

Replace values with #<value># with date - '01-Jan-2015'

I want to replace values in file. Replace : values #startdate# with - '01-Jan-2015' values #code# with - 'SA','CA'
1
vote
1answer
36 views

Need to filter the data on the basis of two column having many to one relationship using awk

I have a big file having 50s columns and 100K of rows delimited by |. Now $2(col 2) has multiple type of $1(col 1) value which means col 2 will be repeated. So I have sorted the file. I need now to ...
1
vote
3answers
51 views

Remove all the rows below a specific row number from one specific column

I've asked a question before on true row deletion, and got some great answers, but I'm now interested to know how you would remove all the rows below a specific row number from one specific column. ...
6
votes
4answers
61 views

True column deletion, not selecting all columns that you want

I know this is a simple question and in many ways has been answered before, however every use i've seen of awk, sed, grep etc have all been selecting and printing the columns you want. In a case where ...
-1
votes
2answers
40 views

how to change record or field separator of a file [on hold]

How can I change the record separator of a file? For example, concatenating multiple lines into a single line, by changing record separator from newline to space. Another example is to split a line at ...
0
votes
1answer
22 views

extract two columns and add at a specific location to another file that is already full of columns

I have a file where I want to extract two columns and add at a specific location to another file that is already full of columns. Could I cat file 1 and make it a variable and then cat file 2 and ...
1
vote
2answers
33 views

I would like to replace a pattern with another pattern found in the same line

I would like to replace the second tab-delimited field with the first underscore delimited field found in the same line Input Cho1_01:C4WMNACXX:1:250326554 1 2 4 2 -9 1 1 4 3 1 4 1 -9 4 1 4 1 3 ...
3
votes
1answer
51 views

Join arbitrary lines to output as one line

I want to join the 3rd and 5th line of a file to the output of a script without trailing new lines. Is it possible to implement this construction by using the code: { echo 'Some text' echo 'Some ...
1
vote
2answers
45 views

add a file name to lines in this file

I have fasta files, coming from different samples eg: 01.fasta, 02.fasta, etc. All of them contain OTU list with the following pattern: >OTU1 CCTACGGGCGGCTGCAGT >OTU2 CCTACGGGTGGCTGCAG I ...
0
votes
1answer
25 views

How to insert fields from one file into fields of another file at particular location?

Is there any quick way to inserts fields from one file into another. For example, if the file contents of file1 and file 2 are like this : File1: ======= size1 size2 size3 File2: ======= char ...
1
vote
2answers
87 views

Grep lines that contain “1111-11-11” or “2382-21-20”. (Any Numbers)

I have tried to do this and I came up with this grep -E '\<[0-9]{4}"-"[0-9]{2}"-"[0-9]{2}\>' It doesn't work and the reason for that is the "-" and multiple grep things, so I tried dividing ...
3
votes
2answers
51 views

Find IP addresses visiting /page1 but not /page2 from nginx access logfile

I need help with selecting specific data's from log files. I need two scripts: I need to select all IP addresses that only visited /page1 I need to select all IP addresses that visited /page1 but ...
1
vote
0answers
21 views

Amend perl script so that words are matched on a word for word basis [migrated]

I have been using this perl script (thanks to Jeff Schaller) to match 3 or more words in the title fields of two separate csv files. Original question here: Matching 3 or more words from fields in ...
1
vote
4answers
29 views

cut column 2 from text file

My text file has no delimiter to specify separator just spaces, how do I cut out column 2 to output file, 39 207 City and County of San Francisc REJECTED MAT = 0 78 412 Cases and ...
7
votes
4answers
8k views

tail -f, but with line numbers

I'm trying to see how many times foo bar appears in /var/log/foo.log within an arbitrary amount of time on a remote server, but nothing that I've tried so far has worked. I've already got a timer ...
1
vote
1answer
26 views

How to append the first line at the end of each line of a file with Sed?

I'm trying to append the first line of a file at the end of each line with a single sed command: Example: line 1 line 2 line 3 line 4 should become: line 1 line 1 line 2 line 1 line 3 line 1 line ...
3
votes
2answers
80 views

Organizing three dimensional data with awk/sed/perl

I have this file (a sparse matrix): PC.354 OTU1 6 PC.354 OTU2 1 PC.356 OTU0 4 PC.356 OTU2 7 PC.356 OTU3 3 I want an output like this (dense matrix -classic .biom table): OTU_ID PC.354 PC.355 ...
3
votes
2answers
36 views

Get a string and place it at the start of a line until the next string is found

I have a big non-standard xhtml file that i'm using sed (about 4 times) to iterate down to just the basics I need to throw the data into a MySQL database. My last bit i'm struggling with. The file is ...
1
vote
1answer
18 views

Library database

I have a database of readers and I would like to lend them a book and write it into the database. Example: Reader 753 Leonardo DaVinci davinci@gmail.com 321654987 Lended Books: grep ...
28
votes
4answers
25k views

Print file content without the first and last lines

Is there a simple way I can echo a file, skipping the first and last lines? I was looking at piping from head into tail, but for those it seems like I would have to know the total lines from the ...
0
votes
3answers
52 views

How to extract logs between the current time and the last 15 minutes

I want to extract the logs between the current time stamp and 15 minutes before and sent an email to the people configured. I developed the below script but it's not working properly; can someone help ...
3
votes
5answers
309 views

Increment index in file

Alright, it seems I cannot find a way to do what I need. Let's say I have a text file A like this (freqBiasL2[27]) (SatBiasL1[27]) (defSatBiasL2_L1[27]) (defSatBiasSlope[27]) ...
0
votes
1answer
99 views

Find string in text file and execute a script depending of the result

I have a file (Data.txt) with something like this: A=1234597890 B=192.168.1.1 sometimes B goes first and sometimes only A or B. So, If I find: A, Print A content and execute A script B, ...
1
vote
2answers
35 views

Adding 'exception' words to the rule of perl matching titles script

I have been using this perl script (thanks to Jeff Schaller) to match 3 or more words in the title fields of two separate csv files as answered here: Matching 3 or more words from fields in separate ...
5
votes
3answers
18k views

Extracting part of lines with specific pattern using awk,sed

I have a question regarding the awk/sed operators. I have a big file which has following set of lines repeated Expression loweWallrhoPhi : sum=-6.97168e-09 Expression leftWallrhoPhi : ...
0
votes
2answers
69 views

Replicate lines in text file according to a column value

I have a text file with a column containing text/data followed by an abundance column, e.g. Car 3 Apple 2 In order to subsample these columns I would like to replicate each text column by ...
0
votes
2answers
25 views

Generating wordlist permutations from existing files

I have two wordlist files; the first contains: John Jerry Jim the second contains: one two three The files are several thousand lines each, not just the 3 words above. How can I generate ...
3
votes
4answers
56 views

Remove a certain word in a file

How do I remove just abcd11.gmail.com and *test1: from line 1 and abcd14.gmail.com and *fortest2: from line 2 and abcd19.gmail.com and *asatesteg: from line 2 from below leaving the remaining as is? I ...
0
votes
3answers
60 views

How can I retrieve and compare two values from a file? [closed]

How can I retrieve and compare two values from the first two lines of a file? I have the following text: 05-24-2016, 2:59:32,0,0 05-24-2016, 2:59:37,0,0 05-24-2016, 2:59:42,0,0 05-24-2016, ...
1
vote
2answers
66 views

Removing spaces from the fields in a semicolon-delimited file using shell script

I have a csv file like this: 03139; 5; IT1234978208; 20150930 ; CTZ 13/31.12.15; 03137; 6; IT1234978206; 20151015 ; CTZ 13/31.11.18; .... .... 03134; 8; IT1234974406; ...
1
vote
1answer
41 views

Sort text by specific location on each line

What I want to achieve is sorting lines of text by a specific point in the text (the lines of text have no spaces or columns). I've worked out a solution to this, but I want to know if there is a more ...
2
votes
1answer
37 views

How to standardize the number of characters in a column?

I have a column (in my data-file database) where the number of characters is not standardized, and need to standardize. How can I do it? Example: Sample_ID 159 148 1110 1100 150 But my bank does ...
3
votes
1answer
42 views

Substitute text fields with values from other file

Alright, I cannot figure out how to do this. I have a fileA which looks like this : ([7]RIMS_ID) : "CNR" (refGain_A[7]) ...
0
votes
1answer
47 views

Using regexp_replace to replace single quotes around string but not within words e.g. can't

I'm using the below code from a bash script to replace single quotes from my title column in a postgres database with spaces. psql -U postgres -d my_database -c "UPDATE my_table SET title = ...
13
votes
9answers
32k views

A command to print only last 3 characters of a string

I know that the cut command can print the first n characters of a string but how to select the last n characters? If I have a string with a variable number of characters, how can I print only the ...
0
votes
0answers
24 views

copying every 2nd line starting from 1 [duplicate]

The below has actually been asked before, which I hadn’t realised. but out of curiosity, how would you go about it if you wanted not just odds or evens, for example every 5th line I'm trying to copy ...
15
votes
12answers
1k views

Delete duplicate lines pairwise?

I encountered this use case today. It seems simple at first glance, but fiddling around with sort, uniq, sed and awk revealed that it's nontrivial. How can I delete all pairs of duplicate lines? In ...
5
votes
4answers
131 views

Build table - Add column depending on filenames

I have a huge amount of files having the following naming style: WBM_MIROC_rcp8p5_mississippi.txt WBM_GFDL_rcp8p5_nosoc_mississippi.txt DBH_HADGEM_rcp4p5_co2_mississippi.txt ...
-1
votes
0answers
21 views

Word processing [closed]

When I backspace in a word doc to erase to the left, the cursor always jumps to the right end of the line and erases. Are all word processors like this and how do I avoid it? This is a real nuisance.
-3
votes
0answers
19 views

Evaluating a Question Generation System [closed]

I'm finishing a System that generates questions automatically from declarative sentences. Example: "The capital of Russia is Moscow" The system may generate questions like: What is the capital of ...
2
votes
3answers
157 views

reformat a text file to have all text on one line using CentOS 5.6

I have a 210-line block of HTML that I have stored in a HTML for debugging purposes. The HTML, as you would expect, is formatted accordingly so is stored in the file with spaces and tabs. Is it ...
1
vote
2answers
53 views

find string from one file in another if not present then remove from original file

I'm trying to make a script that looks through each line of one file and if a line fails to match anywhere in any line of another text file then remove that line from the original file. so far I've ...
-3
votes
0answers
32 views

How to compare an input name (first and last) with names in a file (revised) [closed]

How do you compare an input name (first and last) with names in a file and print out either name is good or name is bad? I can't seem to find this anywhere. Do I need to set up the file a certain way? ...
2
votes
4answers
54 views

Create text corresponding to multiple variable assignments from a single variable

I have a variable that has a value like this line=dog,/path1,/path2 What I want is to create a variable from the value; i.e dog should be the variable and want to get the below output: ...
1
vote
1answer
48 views

How to find match with surrounding words?

I want to find match with surrounding words. I also want it to be fast. Here's what I've got: grep -o -P '(\w+ ){0,n}match( \w+){0,m}' n is number of words before the phrase, m is number of words ...
0
votes
1answer
50 views

sed use from android shell

I am getting familiar with sed on the android terminal emulation: the version I am using the sed included in the toybox which is installed on my unrooted Sony device. sed --version (or toybox sed ...
3
votes
1answer
44 views

Modify values in a column based on number of colons it contains

I would manipulate the contents of the second column based on the number of colons it contains. If the second field contains more than one colon then I would require the content before first colon ...
1
vote
2answers
74 views

Print text between multi-line delimiters

I am trying to extract data between 2 matching patterns, only if the content is not empty and I have issues doing it: here is an example: ============================== Queue Manager is : ...
3
votes
1answer
27 views

Use a list file to extract a second list from a master file (and comment out some lines)…HOW?

I have two files: master.tbl and sites.lst The file master.tbl is a list of sites, with their port configurations, port names, IP addresses, etc. There are lines containing site descriptions, ...
0
votes
1answer
25 views

Each line into a file (XML as part of file)

Here is a sample of the content in the file: <:abc>test file name <:/abc> XYZ 1122 XML content is pretty heavy and big. Under the directory I am working in I want to create a ...