Questions tagged [bioinformatics]
Use this tag for questions relating to common bioinformatics tasks performed on a *nix system. Things like manipulating/converting between standard biological text formats, extracting data of interest from such formats etc.
297
questions
0
votes
3
answers
60
views
Using "sed" to insert text at the end of a field in TSV file
I want to insert characters in TSV files using the sed command with wildcard expressions:
I have a file like this:
Marker Pvalue Trait Chr Pos
S3_16887238 6.172847e-09 Total_Soil_S 3 ...
- 15
0
votes
5
answers
82
views
How to bring the next line to the end of the first line using awk, separated by a comma?
I have downloaded some sequences from a publicly available database in .fa format. I want to generate a .csv file that contains the name of the sequence, and its length, separated by a comma.
This is ...
1
vote
2
answers
77
views
Sorting a FASTA sequence file according to species names using sed
I have a FASTA file of 536 DNA sequences containing multiple species, but it is not necessarily in order according to species name.
My headers with the sequences look like
>COLFG678-14|MZ630002|...
- 143
2
votes
7
answers
131
views
DNA sequence counting
I have a sequence of DNA separated by some white space. I need to remove the white spaces and return the count of the sequence without the white space characters. Any help on how to do that? I am ...
- 23
0
votes
1
answer
50
views
Pipe containing sort and awk writes too many files to disk while taking random lines from files and ultimately fails due to unavailable storage
I have around 500 GB free disk space on my SSD. I'm trying to run an operation on 10 gzipped files (each around 25GB in size). But I keep running out of storage when I do it in a parallel for loop ...
- 257
0
votes
1
answer
39
views
How to copy some but not all of the files in a directory based on patterns in the file name?
I have a long list of files named according to the following pattern:
S901_S1_fastq.gz
S902_S2_fastq.gz
S903_S3_fastq.gz
S904_S4_fastq.gz
etc.
Because these files correspond to different experimental ...
1
vote
2
answers
116
views
Linearizing a fasta file and removing special characters in
I linearized a fasta file using using awk on a remote computer.
when I used nano to open it, it showed that the file had linearized. However when I downloaded the file to my local computer, and I ...
- 33
2
votes
4
answers
311
views
Removing special characters from a fasta file
I recently linearized a fasta file, using awk. The ouput is perfect however the is a caret(^) in my sequence. I want to remove this caret. below is my attempt, any assistance is highly appreciated.
&...
- 33
0
votes
0
answers
213
views
Can't install mamba on my drive: collecting package metadata (repodata.json): / Killed
I am trying to install mamba through conda on my linux terminal on my chormebook. Please help. I need to install this for a final exmam. My professor just tells me look online. I am not a programmer ...
0
votes
1
answer
106
views
For loop in bash to input a new file in directory
For context: I am running Prokka (as a novice bioinformatician with no Prokka and little Bash experience) and I have multiple FASTA files.
I have been told I need to produce a Bash loop that will ...
0
votes
1
answer
33
views
Variable followed by underscore
I have files molecule416_out.pdbqt, where I am using variable i (molecule$i_out.pdbqt) to loop over all files. But it looks like it is reading the variable as i_out instead of just i. Since all those ...
3
votes
3
answers
370
views
Identify strings between patterns and print entire region between pattern if string is found. Perferably using awk
There are similar questions but none of them quite address the issue I have.
Simply put, I need to print every block that contains any string I'm looking for. Each block start line contains: <entry ...
- 877
0
votes
2
answers
39
views
rename the header of multifasta files in bioinformatics
here is my file1.fa(seqID, \t, sequence), it's not a standard fasta format file.
abc MFSKPTKAPLNKGQEAVAKEFFDFLLDPNATEFNISGPGGTGKTFLMSHLIDDTMPA
cef ...
- 59
1
vote
2
answers
287
views
Need to find position of a character in a text/sequence file?
For example I have the sequence below in a text file called sequence_test:
AAAAATGATGATGTGATTAAATTTTTGAGAGTGATATGATATTTTTATAGATATGAGTAGGATAGTATAGATAG
I want to find the position number of all "A&...
- 29
2
votes
1
answer
216
views
sed modifying in place with -i flag
I'm new to GNU sed (running on macOS) and wish to shorten some file headers (~50 K headers, 78.3 Mb).
I'm trying to modify FASTA files in place by specifying a backup extension with the -i flag.
So ...
- 143
1
vote
1
answer
131
views
How to manipulate and extract certain data from a table?
I am quite new to Linux.
I have data like the following, it is in a tbl format
>Feature gnl|XXX|IFEJKLFI_79
locus_tag IFEJKLFI_05549
locus_tag IFEJKLFI_05549
...
- 11
0
votes
3
answers
62
views
To split multiple columns into rows based on delimiter using shell
I have a tab-separated file that looks like this:
cg13201342 F ARNT;ARNT;ARNT;CTSK 3'UTR;3'UTR;3'UTR;TSS1500
cg05269359 F SCN4B;SCN4B;SCN4B;SCN4B 3'UTR;3'UTR;3'UTR;Body
cg06018296 R NEK3;NEK3;...
- 11
8
votes
5
answers
921
views
bash loop to replace middle of string after a certain character
I have 120 files (genomes.faa) that all have headers between each sequence
>GENOME1_00001 HYPOTHETICAL PROTEIN A
NQFTIAQSQVGLEDALLDL
>GENOME1_00002 HYPOTHETICAL PROTEIN B
NQFTIAQSQVGLEDALLDL
&...
- 81
0
votes
6
answers
179
views
rename the header of multifasta files
I have multi-fast files like this
>aaaaa.1_prot_CAA66754.1_10
MEKLLDAYLSSLRLNRRQVSEQTGLNYTTLQRASDKDALMISPRILWGIAMMVDKTPGQVLDELIELEMK
>bbbbb.1_prot_CAA66753.1_11
...
- 43
1
vote
1
answer
47
views
Create a script to run a program that uses multiple input files with the same base name with the exception of one input file
I am new to scripting so I am in need of help.
I'm running a program that is based on four separate test (--max-maf) that requires input files names and a file name for the output. Below is just a ...
1
vote
1
answer
42
views
Looping through thousands of files for big data analysis?
I'm a beginner in bash scripting, can someone please help me here?
In a directory (pamlfiles/), I had list of files (*.fa)
OG0018053.fa
OG0018054.fa
OG0018055.fa
The files look like this:
head -n 2 ....
- 89
0
votes
5
answers
239
views
Creating a sequence with a specific string and varying numbers and letters
I need to create a single row with columns that have a shared string, vary by number, and share repeating letters. My desired output looks like:
SNP1a SNP1b SNP2a SNP2b ... SNP3502a SNP3502b
I am ...
6
votes
1
answer
281
views
awk split a file into multiple files with specific names in another index file
I have a cluster fasta file (called file) which looks like:
>1AB2
>1AB2 AA
NWWIEUNJRNIBGOWNGIOWGRBIGBRGRIOWGI
NCIDHFR8EHGBVPIWOBGIGRI
>1AB3 AA
WNIOREHUEBRGOUERGHBERGIORBGREUGEGO
...
- 375
2
votes
3
answers
192
views
awk/sed split a cluster file in to multiple files
I have a cluster fasta file (called file) which looks like:
>1AB2
>1AB2 AA
NWWIEUNJRNIBGOWNGIOWGRBIGBRGRIOWGI
NCIDHFR8EHGBVPIWOBGIGRI
>1AB3 AA
WNIOREHUEBRGOUERGHBERGIORBGREUGEGO
...
- 375
1
vote
1
answer
116
views
How to install FASTQC on home space as I do not have access to the root server
I am trying to run FASTQC on some data. I am working on a server where I do not have root access. Does anyone know a way where I can install fastqc in my home space as I don't have root on the server ...
0
votes
2
answers
311
views
How to replace the character in one column at the position indicated by another column with the string in a different column
I am rather new to bioinformatics (this is my first post!) and I would appreciate some help on task that has me stuck.
I have a Tab-delimited data table with three columns:
AATTCTTGCA 4 [A/T]
...
1
vote
2
answers
146
views
Group IDs with defined range
I have a sorted file of IDs and numbers (positions). I need to group the positions in the 2nd column into intervals of 500 in a group.
If the values of the row, when compared to the previous row are ...
- 23
1
vote
5
answers
120
views
replace string x by string y according to string z
(EDITED)
I have a file containing quite some problems. For example as below:
Chr1_RagTag_p AUGUSTUS transcript 393571 396143 . + . transcript_id "TCONS_00000070"; gene_id "...
- 35
0
votes
2
answers
93
views
Grouping SNPs as locus based on defined distance
I have a sorted file of IDs and numbers (positions). I need to group the positions in the 2nd column into intervals of 500 and then split into different files.
Input
snp00001 200
snp00002 300
...
- 23
0
votes
0
answers
239
views
Using Process Substitution in docker as input <(command)
Recently, I faced some problems in developing bash pipelines with docker.
This regards to the use of process substitution [<()] as input for a specific docker command.
I know that this is a problem ...
1
vote
6
answers
468
views
How can I remove the portion between the ">" and the word "Bacteria" for each occurrence in a file?
I want to remove the portion between the > and the word Bacteria for each occurrence in a file.
Here is an example of the input:
>AADV02000003.105686.107093 Bacteria;Cyanobacteria;...
2
votes
3
answers
134
views
How to replace a string by using part of the string in the same line?
I have a file like the following on a Linux machine, and the gene_id is wrong. So I would like to replace the string "PB" in the gene_id with the transcript_id from the same line.
$ cat try....
- 35
0
votes
2
answers
40
views
What code to use to see how long Unix took to run a blastn
I have just run the following foreach:
for file in *.fa
do
echo working on $file !
blastn -db ../lotus-date -query $file > $file.blastn
echo finished $file
done
Is there a way to deduce how ...
-5
votes
1
answer
64
views
I want to grep "exonic" in my ".tsv" file provided that column 13 of this file contains "p."
I want to grep exonic in my .tsv file provided that column 13 of this file contains p..
Can you help me with the code I need?
Note: As there are also exonic variants with no amino acid changes, I want ...
-1
votes
4
answers
516
views
How to remove * at the end of each protein sequence in a fasta file
I have a fasta file containing multiple sequences (amino acid sequences) of a particular protein. The last character of the sequences are denoted as "*", which actually represents the stop ...
0
votes
5
answers
746
views
Print subset of lines from file based on column that falls within ranges of second file
I have a file with 4 columns (file1.txt):
chr1 1156 G G
chr1 1157 A A
chr1 1165 T T
chr1 1173 C C
chr1 1175 G G
chr1 1178 T T
...
- 3
0
votes
0
answers
85
views
Compare the second column in file 1 and the second column in file 2 in linux
I have two files, file 1 and file 2. I want to suppose when the chromosome number are the same, if the second column in file 1 is greater than (the second column - 1,000) in file 2 and smaller than (...
- 1
2
votes
5
answers
462
views
How can I extract some values that fall within a range
I have about 4500 lines of aminoacid variations that look like this:
S1437T
H1266Y
T2662A
E1397A
E626K
S1538T
E3021K
Briefly, the letters flanking the numbers are the amino acid residues and the ...
1
vote
3
answers
897
views
Explanation of a sed command
I found this interesting command:
grep -v '^>' test.fasta | tr -d '\n' | sed -e 's/\(.\)/\1\n/g' | sort | uniq -c | sort -rn
I have some grasp what it means (it counts letters from a text file), ...
2
votes
3
answers
488
views
using sed to capture groups
I have a file that looks like that:
chr1 3143567 3143568 .3-2704 1.000000|ENSMUSG00000102693.2
chr1 3143599 3143600 .3-2705 1.000000|ENSMUSG00000102693.2
chr1 3143631 3143632 .3-2706 1.000000|...
- 297
-2
votes
1
answer
317
views
grep command to grep the gene list in first file present in the gff file
I am using this command to grep the gene list in first file present in the gff file.
grep -w -f upregulated_genes_in_BEg GCA_900659725.1_ASM90065972v1_genomic.gff
it did not work for me.. please any ...
2
votes
1
answer
37
views
Loop through file with unique identifer and compute cumulative sum
I have a large text file, as shown below, where column1-colum3 define a region and fourth column is unique identifier. The column 6 is assigned value of the region.
chr6 26204834 26204839 ...
- 105
0
votes
0
answers
205
views
how to remove line breaks in a (.tsv) file
I have a large square correlation matrix file (110,000 RXC) for human transcript data in .tsv format (File name = corr_trans.tsv). When I tried to extract correlation data for a transcript (say ...
- 167
0
votes
3
answers
409
views
how to print file name and total number of fasta sequences?
I have a fasta file namely test.fasta, pas.fasta, cel.fasta as shown below
test.fasta
>tile
ATGTC
>259
TGAT
pas.fasta
>ta
ATGCT
cel.fasta
>787
TGTAG
>yog
TGTAT
>In
NNTAG
I need to ...
- 129
1
vote
3
answers
304
views
How can I add a new element to each line of a file
I have a file with this content:
CHROM_POS
chr10_100009635
chr10_100187980
chr10_100229692
chr10_100267650
chr10_100269675
chr10_100279430
chr10_100285899
I want to add new content to this file like:
...
- 27
0
votes
3
answers
59
views
extract certain substring
I have a file like this:
RSID1 RSID2
chr1_169894240_G_T_b38 chr1_169894240_G_T_b38
chr1_169894240_G_T_b38 chr1_169891332_G_A_b38
chr1_169891332_G_A_b38 chr1_169891332_G_A_b38
chr1_169661963_G_A_b38 ...
- 27
1
vote
1
answer
221
views
simulating user input in a bash script on slurm
I believe my problem is similar as found here but I was confused by the reply (and I can't comment in this question yet due to my low rep).
I have a bash script I need to run using slurm, so when the ...
0
votes
4
answers
222
views
How to separate fields in a column (into their own named columns) and then remove repeated words from each entry
I have a tsv file with 9 variables, as follows:
> seqnames start endwidth strand metadata X.10logMacsq annotation distanceToTSS
The metadata column contains information I want to do ...
6
votes
5
answers
256
views
Remove all sub-fields in column-organized datafile that contain "_XX"
I have this type of file
#CHROM POS ID REF ALT QUAL FILTER INFO
chr1 69511 rs2691305 A G . PASS AC=70482;AN=83312;AF=0.846001;popmax=eas;...
- 61
1
vote
2
answers
258
views
How to ignore part of a filename
Sorry if this question has been asked before. I am new to all of this.
I would like to concatenate all files from different folders that contain R1 at a specific position in their filenames. My ...
- 11