Manipulation of text by programs, scripts, etc.

learn more… | top users | synonyms

2
votes
1answer
29 views

Insert string or line after last instance of a specific search variable, in a loop

I'm wondering if someone could help me with a specific coding question. I have a DNA sequencing file that reads something like this (as an example): ...
1
vote
2answers
64 views

Bash script - chomp parts out of a line

I have a data file (see sample three lines below) separated by pipes. What I want to be able to do is to "chomp" out the fields and put them into variables. They are of pre-defined lengths. I want to ...
3
votes
1answer
41 views

comparing values in separate columns

I've got a csv that looks like this, (there are other columns but these are the pertinent ones) region, city, actual rating, predicted rating NSW, SYD, 0.765, 0.5 Victoria, Melbourne, 4.81225, 5.0 ...
0
votes
1answer
36 views

extract the data from 2 files

I have two files, the first file contain only 1 field, the second file has more lines than first file and also it has different field in each line. File 1: 5 15 20 File 2: 3 5 O 1.25 2.30 3.75 ...
0
votes
1answer
43 views

Removing new line characters around patterns

I have a fasta file of millions of paired sequences, that looks like this: >7001289F:56:HKH3FBCXX:2:1101:1692:2074 1:N:0:CGATGT ...
0
votes
2answers
62 views

How to get only certain fields from a given file?

I have list of employees in following format Name: Phone No: Email: Name: Address: Phone No: Email: Name: Country: Address: Phone No: Email: Name: Address: Name : Email: Address: All I want ...
4
votes
3answers
140 views

Insert text at specific line number

I'm working on a bash script that will split the contents of a text document depending on the data in the line. If the contents of the original file were along the lines of 01 line 01 line 02 line ...
3
votes
2answers
55 views

sed pattern and multiline substitution

I need to parse a text file and replace a certain portion of it. I believe sed is the way to do it but not sure how to handle multiple lines with it The pattern I am looking is set cells { \ cell1 ...
1
vote
4answers
105 views

Bash: How to print additional column matching IP Address to Name

I'm using netstat to collect information about connected IP's. And I have the following format: netstat -tn 2>/dev/null | awk '/:80 / {print $5}' | sed 's/.*::ffff://' | sed 's/:.*//' | sor t | ...
3
votes
2answers
47 views

Sorting by two columns

I need help with this homework assignment: Write a shell script named sales1 using nawk and sort commands to do the following: 1. print the file in two sections, east and west, with lines alphabetized ...
1
vote
1answer
78 views

What does :<i> code stand for in subtitles?

I downloaded a subtitle from subscene http://subscene.com/subtitles/the-5th-wave/english/1298314 and it contains some coding such as :<i> and </i> which is not being ...
1
vote
1answer
57 views

Extract random sample of N lines based on pattern

I have a file formatted like this: train/t/temple/east_asia/00000025.jpg 94 train/t/temple/east_asia/00000865.jpg 94 ... train/s/swamp/00000560.jpg 92 train/s/swamp/00000935.jpg 92 .... ...
2
votes
2answers
26 views

How to append a string to the first column of a matrix file?

I have a matrix file as follows. The first column is a name list. I want to add _i1 to the end of each name. A1A A1B A1C A1D A1E TR6764_c0_g2 0.00 0.02 0.00 0.00 0.00 ...
4
votes
4answers
109 views

Replace strings in a file based on a list of strings and a list of corresponding replacements

I am trying to replace strings in a file A: Hello Peter, how is your dad? where is mom? where the strings to be replaced are in file B: Peter dad mom and their corresponding replacements are in ...
0
votes
0answers
12 views

Linking and combining text from different documents into dynamic document

I am taking notes on many, related topics from multiple sources. For instance, one of the sources is a textbook on machine learning. I type the notes in an .odt document (but whatever, it could be ...
3
votes
2answers
51 views

On applying commands to groups of lines from stdin

The Unix package datamash supports the application of several summarizing operations to groups of input lines. For example1, here datamash is used to compute the sums of column 2 for each value in ...
1
vote
1answer
35 views

Removing columns from a file

I'm trying to remove columns 75 through 85. The colrm command works, but the problem is that it stops whenever it finds an extended ASCII character in the file. So, my question is this: is there ...
1
vote
1answer
42 views

Compare independent lines in two files [closed]

I have two files, each with 500k+ user accounts. I need to find the differences between the two (the missing accounts/delta). Each file is in a simple format, each line contains an email address in ...
3
votes
1answer
40 views

Get contents of a file from first instance to last instance of some arbitrary string

How can I view in less from the first instance of some arbitrary string "foo" to the last instance? This arbitrary string will be on most every line of the log. I don't want to do grep "foo" bar.log ...
1
vote
1answer
37 views

Sed delete line if nth character is not specific character

I have a csv file, and I want to delete lines if their 12th character is not ;. So, for example, my file looks like this: 2266308;A;B;dfsgsfdg 2266309;A;BJHSADFK;gfsdg 2266310;A;B;dfg and I ...
2
votes
2answers
67 views

Split File into 10 files with unique item in each file and maximum line number limit in each file

I have a huge file (more than 2M records in it). Here is my requirement: Initially, break the one big file up in to 10 smaller files. The files should be formatted as follows: ...
1
vote
1answer
43 views

How to set csplit output filenames by pattern?

Is there a way to set output names of csplit through the pattern? I want to split the ssh-config by hosts to corresponding files. If there is a solution without csplit I'm happy as well.
0
votes
2answers
43 views

Bash Script with Multiple Lines and Grep

I have output from a specific command that can provide the number of times an application has been marked as 'used' on an OS X system. The command is mdls. The output looks similar to the following ...
0
votes
1answer
42 views

subset a file just on one column

I have a tab delim file, I want to make a subset on the basis of column 2, choosing "A". 435 A 2 27.92464882 8.63E-07 1 752721 564 F 2 30.12532071 2.87E-07 1 798959 654 A 2 ...
0
votes
1answer
30 views

sed find line in file and remove characters

I have a file that contains lines like this proto udp . . . .. server 10.10.8.0/24 255.255.255.0 I need a way to remove only /24 from this line server 10.10.8.0/24 255.255.255.0 to become server ...
0
votes
2answers
60 views

How do I find a range of numbers in the content of file and move files containing the range of numbers to another directory?

Okay so I have about 90 files, each with a body like 187942 78.9 I want to filter files that has the values between 70 to 90 on the second column and move them to another directory. I'm ...
-2
votes
3answers
97 views

Extract sequences from a fasta file

I have a fasta file (not in right format) that contains hundreds of thousands of different lengths of DNA sequences like this: ...
3
votes
2answers
58 views

match columns and replace

I am newbie in linux. I have two very large text files with space-delimited fields: File1 527858 51 2 27.92464882 8.63E-07 570289 82 2 30.12532071 2.87E-07 571034 90 2 ...
1
vote
2answers
50 views

How to add strings from a name list file to another file?

I have a fasta file (seq.fa) which is a standard file format for genetic info, like so: >TR1|c0_g1_i1 GTCGAGCATGGTCTTGGTCATCTTCCTTTCAAAGAA >TR6|c0_g1_i1 GTGGAATATCGCCAGTGACCATCACTGATTAACCTG I ...
1
vote
3answers
73 views

How to do following formatting using sed or awk or bash script on linux

Suppose, I have two files: file1 having below contents. message_aaa.wav test 1 of 1 test Label : aaa test Sentence : test aaa test String: test message Interpretation ...
1
vote
2answers
54 views

AWK: How to put field ($1) inside regular expression to select all prefixes?

Assume I have a file A: fileA fileB Suppose I have now a file named: fileA_someprefix_20160101.txt Now I want to match all lines from A which prefix this filename, so I thought: ...
0
votes
2answers
55 views

Transforming a list of files separated by new lines into a variable

Goal: I'm trying to find all instances of *.clj or *.cljs files recursively within a directory, store them in a string variable (separated by new lines) and then transform them. So if the following ...
-1
votes
1answer
24 views

Sorting and outputting in another file

How can I sort a file then output it into another file I never created or do I have to create the file I want to put in. Might not make sense how I am typing it so... Let's say the file I have is ...
4
votes
3answers
114 views

String replacement using a dictionary

What is a good way to do string replacements in a file using a dictionary with a lot of substituend-substituent pairs? And by a lot, I actually mean about 20 – not much, but many enough so that I want ...
0
votes
3answers
82 views

What is the most concise script to list the missing ones in a digit sequence? [duplicate]

For example, we have the content 001 002 004 008 010 in a text file named file, how to extract the missing 3 5 6 7 9?
0
votes
2answers
58 views

Find and replace words in text file recursively

I have tree of folders inside these folders there are many text files. These files have the same name "log.txt" and the same format. For example inside every file we can see multiple lines like: date ...
1
vote
1answer
31 views

pattern regex for apache rewrite rule

I have the following pattern on a few thousand lines I will like to use regex to convert to what I want RewriteRule ^/omg/deadly-venom/?$ http://www.example.com/omg/deadly-venom [L,NC,R=301] ...
1
vote
1answer
47 views

Why don't the SHA's match?

I am trying to write a script and it uses the SHA of a date but I am getting two different results and for the life of me can't figure out why. echo -n 03112016 | cut -d'.' -f4 | sha256sum | cut -d' ...
3
votes
3answers
41 views

Strings - Extract File Version Recursive

I have been using strings to extract and compare file versions between directories. I don't seem to be able to do this recursively. The method I have been using per file is: strings -e l ...
-1
votes
3answers
79 views

Better way to parse string with bash?

I need to parse the below command to extract namespace & interface names using a bash script. I generally do this using IFS, but that seems tedious to me. Is there a better way of doing this ...
-1
votes
2answers
36 views

First and last five lines of file [duplicate]

How to see first and last 5 lines of file? Right now I am using head filename ; tail filename Any other way around and more efficient?
1
vote
4answers
54 views

Remove any occurrence of '-' from a field in shell script

below is my csv file,I want to remove all occurrences of - from retailer_id field and create new csv. Kindly help... ...
0
votes
2answers
61 views

Compare a file with a column of another file

I have a file File1 with data as aaa bbb And another file File2 with data as: 2,aaa,234 w,bbb,589 4,ccc,675 I need to compare the File1 data with column2 data of File2 and print the matching ...
0
votes
3answers
55 views

Awk command is not working properly

I have a very simple problem, but for reasons it does not work properly.. I have these .txt files in the following format 2 250 1 4 250 1 5 250 1 I wanted to subtract 1 from the ...
-1
votes
3answers
72 views

Change all values with an absolute value less than 1 with 0

I have a large file with a lot of rows and columns; the value of the contents are between -2 and 2. I'm interested in replacing values which, in their absolute value form, are less than; I want to ...
3
votes
5answers
75 views

How to extract data from a file and construct a filename from one of the values

I'm trying to pull two values from a configuration file and write them to a new file. The hard part is that I want the name of the new file to be determined by one of the values in the input file. ...
1
vote
2answers
38 views

Joining columns from files if they contain a match in another column

I have two files: File1.txt 30 40 A T match1 string1 45 65 G R match2 string2 50 78 C Y match3 string3 File2.txt match1 60 add1 50 add2 ...
3
votes
6answers
170 views

Add quotes and new delimiter around space delimited words

I have the following string y10_zcis y10_nom y10_infl y20_zcis y20_infl y30_zcis I would like to transform this to "y10_zcis", "y10_nom", "y10_infl", "y20_zcis", "y20_infl", "y30_zcis" I ...
1
vote
1answer
47 views

How to test if certain strings existing in file are present in another file?

I have a file sites.txt that is generated by a script. It's a list of host names. I have another list of host names sites_old.txt. I need to check whether any of the host names present in the ...
1
vote
2answers
34 views

Reorganizing lines in a text file

So I have the following file: First Line Second Line Third Line 3 0.374 2 0.777 1 0.245 Fourth Line 4 0.687 ... And I want to reorganize it so the lines beginning with an int are appended to the ...