Skip to main content

Questions tagged [bioinformatics]

Use this tag for questions relating to common bioinformatics tasks performed on a *nix system. Things like manipulating/converting between standard biological text formats, extracting data of interest from such formats etc.

Filter by
Sorted by
Tagged with
2 votes
5 answers
271 views

Grouping rows by categories avoiding repetition

I have a tab-separated file with two columns on a Linux machine. The first column contains names, the second column contains GO IDs (these are always of the format GO: followed by seven digits) ...
rseg's user avatar
  • 35
5 votes
6 answers
249 views

subset columns from the 1st file using column names in 2nd file

I have two text files: 1st file is a Tab delimited file which looks like this: chrom pos ref alt a1 a2 a3 a4 10 12345 C T aa bb cc dd 10 12345 C T aa bb cc dd 10 12345 C ...
user3138373's user avatar
  • 2,559
2 votes
5 answers
974 views

Replace new lines with spaces using awk

I have a text file that I generated of all files in a directory. I'd like to use this file as input into a script that I have, but I need the text file to be formatted in a particular way to be parsed ...
lovelyrubbish's user avatar
1 vote
5 answers
110 views

sed command to replace a word within a line following a pattern

I'm working with a file that looks like the following, containing with over 50,000 lines of gene IDs followed by their sequence: gene_A:3342234 CTCTTTCTTTTACGCCT gene_A:1244-5205 CTCTTTCTTTTACGCCT ...
bryophyta's user avatar
2 votes
5 answers
446 views

How to split a given column's string values in a text file

I have a text file on a Linux machine with two columns: Column 1 = id_no (most are 5, with some 6 digits long); Column 2 = genetic_markers (all are 50674 digits long); 12345 0102010205 54322 ...
Michiel Van Niekerk's user avatar
1 vote
4 answers
149 views

Remove everything in a third column but only keep specific text

I have a data set with three columns: https://drive.google.com/file/d/1gtCssfAXHxRjGfX8uTAaimGPWCA2cnci/view?usp=sharing Here are the first few lines: ID transcript_id go_description ...
Muahammad Ahmad's user avatar
1 vote
4 answers
344 views

Retrieve the 1st and 5th column of a tab-separated file, convert the spaces in the 5th to tabs

I have a tsv file with tab-separated columns. I want to obtain the 5th column, which has space-separated values. Convert the space-separation to tab-separation and save as a new file. Attempt: cut -d&...
Anon's user avatar
  • 133
1 vote
5 answers
86 views

How to count word from a column when consecutive cells are equal in a different column using shell script!

I'm trying to count the number of C_R and S_R in column 9 when consecutive cells in column 2, column 3, and column 1 are the same. The file is in bed format (tab-separated format). The original file ...
Debajyoti Kabiraj's user avatar
1 vote
4 answers
193 views

Duplicate part of a line to another part

I would like to copy the first part (IxoscaEVm****t1_, without the '.p[number]') of the line starting with ">" and paste a before the last ":" of the same line. Input: >...
alex kiarie's user avatar
0 votes
1 answer
42 views

Counting characters between grep searches

Is there a way I can use the grep command in conjunction with a series of other commands to find a character sequence (ie 'GAATTC' in a fasta file) and count how many characters are between each match?...
Alina's user avatar
  • 1
1 vote
3 answers
95 views

awk doesnt add columns to tab delim file

I'm using the following code to add two new columns (15 and 16) to a tab delim txt file based on calculations from other existing columns. Problem: new column data is shown in terminal but file is not ...
stephen ramnarine's user avatar
0 votes
2 answers
105 views

AWK: If two key columns match between files, add the 16th column of one file to the matching row of the other while keeping non-matching rows

I have two tab separated files (FileA.tsv and FileB.tsv). FileA.tsv id graph circle several columns... length 196-0 196 0 ---- 12874 195-1 195 1 ---- 12874 56-0 56 0 ---- 3349 115-1 115 1 ---- ...
JAL's user avatar
  • 3
5 votes
5 answers
835 views

How to convert all text into 1 except "0"s and first two fields in a csv file?

I have several large.csv files that I'd like to convert to binary (1 and 0) format. In which all cells containing text would become 1 and 0 would remain 0, except first two fields. head Test.csv Iss1,...
Debajyoti Kabiraj's user avatar
0 votes
0 answers
325 views

merging from 1022 files open Too many open files error?

I use samtools for coordinate sort a bam file. This command creates lots of files while running. I run the following command samtools sort -o sorted.bam genomic_dataset.bam But it gives the following ...
heyula's user avatar
  • 21
1 vote
2 answers
55 views

Change irregular header with extra whitespace

I am working with data in this obscure file format: SNP A1 A2 F1 I1 F2 I2 F3 I3 rs0001 A C 0.02 0.00 1.99 (Note the whitespace around the first three fields) The header is long (500k ...
Giulio Centorame's user avatar
2 votes
4 answers
205 views

Remove specific character strings from text file using sed, no change to output file?

Edit: added block quote text I have a tab delimited text file (acc.paired.txt) of illumina sample names (head): SRR10598163_R1.fastq.gz SRR8916417_R2.fastq.gz SRR10598049_R1.fastq.gz SRR10598163_R2....
Geomicro's user avatar
  • 123
1 vote
2 answers
83 views

filter lines based on some criteria

I have some .vcf files and I want to filter some variants out. This is just small part of my file: there are some header lines at the beginning of the file (starting with ##) and then variants (one ...
Anna1364's user avatar
  • 1,036
1 vote
5 answers
325 views

Sed - match up to, but not including, pattern, with newlines included

I have an input that is in a multi-fasta format. Essentially: >header1 AAAAAATTTTCCCCGGGG AAAATGTGTGTCTCTCTC ATAGGATGAGT >header2 GGGGTGTGTGATGTAGTA AAAATGTGTGTCTCTCTC ATAGGATGAGT .... >...
Laura's user avatar
  • 199
0 votes
1 answer
31 views

sort by two numerical columns

I am trying to sort a genetic file based on chr (column 2) and position (column 3) that both start from lower to higher. my table is like this in large file SNP CHR BP A1 A2 effect_allele_frequency ...
motor 30's user avatar
0 votes
3 answers
193 views

Using "sed" to insert text at the end of a field in TSV file

I want to insert characters in TSV files using the sed command with wildcard expressions: I have a file like this: Marker Pvalue Trait Chr Pos S3_16887238 6.172847e-09 Total_Soil_S 3 ...
Vinod Kumar's user avatar
0 votes
5 answers
158 views

How to bring the next line to the end of the first line using awk, separated by a comma?

I have downloaded some sequences from a publicly available database in .fa format. I want to generate a .csv file that contains the name of the sequence, and its length, separated by a comma. This is ...
Hrishikesh Hardikar's user avatar
1 vote
2 answers
215 views

Sorting a FASTA sequence file according to species names using sed

I have a FASTA file of 536 DNA sequences containing multiple species, but it is not necessarily in order according to species name. My headers with the sequences look like >COLFG678-14|MZ630002|...
compbiostats's user avatar
2 votes
7 answers
299 views

DNA sequence counting

I have a sequence of DNA separated by some white space. I need to remove the white spaces and return the count of the sequence without the white space characters. Any help on how to do that? I am ...
user556983's user avatar
0 votes
1 answer
58 views

Pipe containing sort and awk writes too many files to disk while taking random lines from files and ultimately fails due to unavailable storage

I have around 500 GB free disk space on my SSD. I'm trying to run an operation on 10 gzipped files (each around 25GB in size). But I keep running out of storage when I do it in a parallel for loop ...
paropunam's user avatar
  • 257
0 votes
1 answer
92 views

How to copy some but not all of the files in a directory based on patterns in the file name?

I have a long list of files named according to the following pattern: S901_S1_fastq.gz S902_S2_fastq.gz S903_S3_fastq.gz S904_S4_fastq.gz etc. Because these files correspond to different experimental ...
Clayton Tracey's user avatar
1 vote
2 answers
359 views

Linearizing a fasta file and removing special characters in

I linearized a fasta file using using awk on a remote computer. when I used nano to open it, it showed that the file had linearized. However when I downloaded the file to my local computer, and I ...
thole's user avatar
  • 33
2 votes
4 answers
2k views

Removing special characters from a fasta file

I recently linearized a fasta file using awk. The output is perfect. However there is a caret(^) in my sequence. I want to remove this caret. below is my attempt, any assistance is highly appreciated. ...
thole's user avatar
  • 33
0 votes
1 answer
811 views

For loop in bash to input a new file in directory

For context: I am running Prokka (as a novice bioinformatician with no Prokka and little Bash experience) and I have multiple FASTA files. I have been told I need to produce a Bash loop that will ...
notknowingyet's user avatar
0 votes
1 answer
87 views

Variable followed by underscore

I have files molecule416_out.pdbqt, where I am using variable i (molecule$i_out.pdbqt) to loop over all files. But it looks like it is reading the variable as i_out instead of just i. Since all those ...
Jatin Kashyap's user avatar
3 votes
3 answers
508 views

Identify strings between patterns and print entire region between pattern if string is found. Perferably using awk

There are similar questions but none of them quite address the issue I have. Simply put, I need to print every block that contains any string I'm looking for. Each block start line contains: <entry ...
Giles's user avatar
  • 887
0 votes
2 answers
74 views

rename the header of multifasta files in bioinformatics

here is my file1.fa(seqID, \t, sequence), it's not a standard fasta format file. abc MFSKPTKAPLNKGQEAVAKEFFDFLLDPNATEFNISGPGGTGKTFLMSHLIDDTMPA cef ...
Wilson D's user avatar
1 vote
2 answers
570 views

Need to find position of a character in a text/sequence file?

For example I have the sequence below in a text file called sequence_test: AAAAATGATGATGTGATTAAATTTTTGAGAGTGATATGATATTTTTATAGATATGAGTAGGATAGTATAGATAG I want to find the position number of all "A&...
user19194612's user avatar
2 votes
1 answer
483 views

sed modifying in place with -i flag

I'm new to GNU sed (running on macOS) and wish to shorten some file headers (~50 K headers, 78.3 Mb). I'm trying to modify FASTA files in place by specifying a backup extension with the -i flag. So ...
compbiostats's user avatar
1 vote
1 answer
483 views

How to manipulate and extract certain data from a table?

I am quite new to Linux. I have data like the following, it is in a tbl format >Feature gnl|XXX|IFEJKLFI_79 locus_tag IFEJKLFI_05549 locus_tag IFEJKLFI_05549 ...
ahmedbar's user avatar
0 votes
3 answers
529 views

To split multiple columns into rows based on delimiter using shell

I have a tab-separated file that looks like this: cg13201342 F ARNT;ARNT;ARNT;CTSK 3'UTR;3'UTR;3'UTR;TSS1500 cg05269359 F SCN4B;SCN4B;SCN4B;SCN4B 3'UTR;3'UTR;3'UTR;Body cg06018296 R NEK3;NEK3;...
 Heba's user avatar
  • 11
8 votes
5 answers
1k views

bash loop to replace middle of string after a certain character

I have 120 files (genomes.faa) that all have headers between each sequence >GENOME1_00001 HYPOTHETICAL PROTEIN A NQFTIAQSQVGLEDALLDL >GENOME1_00002 HYPOTHETICAL PROTEIN B NQFTIAQSQVGLEDALLDL &...
Goodolgab's user avatar
0 votes
6 answers
467 views

rename the header of multifasta files

I have multi-fast files like this >aaaaa.1_prot_CAA66754.1_10 MEKLLDAYLSSLRLNRRQVSEQTGLNYTTLQRASDKDALMISPRILWGIAMMVDKTPGQVLDELIELEMK >bbbbb.1_prot_CAA66753.1_11 ...
loki's user avatar
  • 43
1 vote
1 answer
53 views

Create a script to run a program that uses multiple input files with the same base name with the exception of one input file

I am new to scripting so I am in need of help. I'm running a program that is based on four separate test (--max-maf) that requires input files names and a file name for the output. Below is just a ...
geneteics_diva's user avatar
1 vote
1 answer
71 views

Looping through thousands of files for big data analysis?

I'm a beginner in bash scripting, can someone please help me here? In a directory (pamlfiles/), I had list of files (*.fa) OG0018053.fa OG0018054.fa OG0018055.fa The files look like this: head -n 2 ....
sunnykevin's user avatar
0 votes
5 answers
986 views

Creating a sequence with a specific string and varying numbers and letters

I need to create a single row with columns that have a shared string, vary by number, and share repeating letters. My desired output looks like: SNP1a SNP1b SNP2a SNP2b ... SNP3502a SNP3502b I am ...
Zoe Diaz-Martin's user avatar
6 votes
1 answer
532 views

awk split a file into multiple files with specific names in another index file

I have a cluster fasta file (called file) which looks like: >1AB2 >1AB2 AA NWWIEUNJRNIBGOWNGIOWGRBIGBRGRIOWGI NCIDHFR8EHGBVPIWOBGIGRI >1AB3 AA WNIOREHUEBRGOUERGHBERGIORBGREUGEGO ...
Jiao's user avatar
  • 375
2 votes
3 answers
244 views

awk/sed split a cluster file in to multiple files

I have a cluster fasta file (called file) which looks like: >1AB2 >1AB2 AA NWWIEUNJRNIBGOWNGIOWGRBIGBRGRIOWGI NCIDHFR8EHGBVPIWOBGIGRI >1AB3 AA WNIOREHUEBRGOUERGHBERGIORBGREUGEGO ...
Jiao's user avatar
  • 375
1 vote
1 answer
280 views

How to install FASTQC on home space as I do not have access to the root server

I am trying to run FASTQC on some data. I am working on a server where I do not have root access. Does anyone know a way where I can install fastqc in my home space as I don't have root on the server ...
bioinformatic_beginner's user avatar
0 votes
2 answers
551 views

How to replace the character in one column at the position indicated by another column with the string in a different column

I am rather new to bioinformatics (this is my first post!) and I would appreciate some help on task that has me stuck. I have a Tab-delimited data table with three columns: AATTCTTGCA 4 [A/T] ...
Zoe Diaz-Martin's user avatar
1 vote
2 answers
376 views

Group IDs with defined range

I have a sorted file of IDs and numbers (positions). I need to group the positions in the 2nd column into intervals of 500 in a group. If the values of the row, when compared to the previous row are ...
austin7923's user avatar
1 vote
5 answers
141 views

replace string x by string y according to string z

(EDITED) I have a file containing quite some problems. For example as below: Chr1_RagTag_p AUGUSTUS transcript 393571 396143 . + . transcript_id "TCONS_00000070"; gene_id "...
zzz's user avatar
  • 35
0 votes
2 answers
94 views

Grouping SNPs as locus based on defined distance

I have a sorted file of IDs and numbers (positions). I need to group the positions in the 2nd column into intervals of 500 and then split into different files. Input snp00001 200 snp00002 300 ...
austin7923's user avatar
0 votes
0 answers
389 views

Using Process Substitution in docker as input <(command)

Recently, I faced some problems in developing bash pipelines with docker. This regards to the use of process substitution [<()] as input for a specific docker command. I know that this is a problem ...
Luis Arge's user avatar
1 vote
6 answers
713 views

How can I remove the portion between the ">" and the word "Bacteria" for each occurrence in a file?

I want to remove the portion between the > and the word Bacteria for each occurrence in a file. Here is an example of the input: >AADV02000003.105686.107093 Bacteria;Cyanobacteria;...
Callahan McGovern's user avatar
2 votes
3 answers
344 views

How to replace a string by using part of the string in the same line?

I have a file like the following on a Linux machine, and the gene_id is wrong. So I would like to replace the string "PB" in the gene_id with the transcript_id from the same line. $ cat try....
zzz's user avatar
  • 35

1
2 3 4 5
7